QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. Center for Integrated Cognitive Systems A Vision and Collaborative Proposal
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Center for Integrated Cognitive SystemsA Vision and Collaborative Proposal
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
2
Engineering Method for RAIR Lab’sNext-Generation Logic-based Artificial
Intelligence
• Isolate and dissect human ingenuity.• Mathematize a weak correlate to this ingenuity courtesy of advanced logical systems.• Implement this correlate in working programs.• Augment the correlate with machine-specific power.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3
Selected Research Projects in RAIR Lab
• Integrated Cognition via PERI and “Psychometric AI”
– PERI is a robot with human-level intelligence.– http://www.cogsci.rpi.edu/peri
• The Slate System: Intelligent Assistant in support of Intelligence Analysis and Beyond
– Slate is developed to help analysts @ NSA, CIA, etc.
• Formalizing and Modeling Adversarial Beliefs and Ethical Codes in Support of Wargaming
– predict what the enemy is going to do in modern-day asymetrical conflict
• New World Records in the -Cracking Project: A New-Millennium Attack
• Learning by Reading! -- and the MARMML Machine Reasoning System
– This project is deveoted to building a machine system capable of learning by reading.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
4
Mind is What Brain Does
• How do we think? Solve problems? Learn new things? Remember old things?
• What cognitive processes are involved in reading a book? Recognizing your Grandmother? Writing a letter? Designing a scientific experiment?
• How many things can we pay attention to at once? How does the design of computer software affect the way we think?
• What is cognition in the world? How does it differ from cognition in the head?
• Can we design our environment to facilitate thought? So the tools we use become “invisible” and we can get on with the task at hand?
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
5
What is Cognitive Science? (A 2-slide introduction)
• Cognitive science is the scientific study of mind; more specifically, the systematic attempt to investigate, understand, simulate, and replicate cognitive systems in computational, representational, and dynamical frameworks
• It is a rapidly growing interdisciplinary field that brings together researchers from parts of six fields: artificial intelligence (AI), psychology, philosophy, education, linguistics, and neuroscience
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
6
What is Cognitive Science? (A 2-slide introduction)
• Cognitive scientists use empirical, formal, and computational methods
• Cognitive Science views the mind as a cognitive system that receives, stores, retrieves, transforms, and transmits symbols or representations
• The operations performed on these symbols and representations are termed information processes. Hence, most of cognitive science can be understood as the application of information technology (and information science) to problems in perception, attention, motor control, memory, language, problem solving, and reasoning
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
7
What is Computational Cognitive Science?
• Computational Cognitive Science is what Rensselaer Polytechnic Institute excels at!
• CCS– is the application of cognitive science theory to
human factors practice– seeks to build systems that monitor, support,
aid, and extend human cognitive processes– includes the engineering of artificially intelligent
systems that capture, in computation, human-level “smarts”
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
8
What is Cognitive Engineering?
• Cognitive Engineering is what Rensselaer Polytechnic Institute excels at!
• Cognitive engineering is the application of cognitive science theory to human factors practice
• Cognitive engineering seeks to build systems that monitor, support, aid, and extend human cognitive processes
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
9
Who Supports Cognitive Engineering?
• Academic work in cognitive engineering is primarily supported by:– Air Force Office of Scientific Research (AFOSR)– Advanced Research & Development Agency
(ARDA -- consortium of Intelligence Agencies)– Army Research Institute– Defense Advanced Research Projects Agency
(DARPA)– Office of Naval Research (ONR)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
10
Who Supports Cognitive Engineering?
• Government work in cognitive engineering is primarily supported by:– Air Force Research Laboratories
• Human Effectiveness Directorate (Mesa, AZ & Wright-Patterson, AFB, Ohio) &
• Information Directorate (Rome, NY)
– Advanced Research & Development Agency (ARDA -- consortium of Intelligence Agencies)
– Army Research Laboratory (Aberdeen, MD)– Naval Research Laboratories (DC)– Naval Underwater Weapons Center (Newport,
RI)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
11
Who Supports Cognitive Engineering?
• Leading Industrial Labs in cognitive engineering include:– PARC (formerly XEROX PARC)– Microsoft Research Laboratories– Sandia Laboratories– BBN– IBM Watson– MITRE
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
12
Center for Integrated Cognitive Systems & the Air Force Research
Laboratories• Building on the foundation of excellence at
RPI Air Force funds will enable CICS to become internationally renown
• This will be achieved by– Support for leading edge basic research– Hiring additional post-doctoral researchers and
faculty– Attracting and supporting a cohort of next-
generation researchers (i.e., grad students)– Enmeshing CICS researchers in the challenges
and opportunities provided by AFRL projects
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
13
Center for Integrated Cognitive Systems & the Air Force Research
Laboratories• CICS can provide AFRL with
– Access to leading-edge research and leading-edge researchers
– Increase the awareness within the cognitive community of the types of problems encountered by Air Force researchers
• Annual Spring symposium that brings the best and brightest of the cognitive community together to contemplate Air Force problems
• Fall colloquium alternating between Troy and Syracuse each year
– Advanced degree program• MS in Cognitive Science tailored to AFRL schedules and needs• PhD program that takes into account AFRL constraints
– Student intern program -- Rensselaer students who are supported by Air Force funds will be required to spend a summer or semester working at the Rome lab
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
14
Center for Integrated Cognitive Systems & the Air Force Research
Laboratories• Working together to
– Advance the foundations of Cognitive Science– Bridge the gap between theory and application– Target Air Force needs by incorporating leading
edge cognitive theory into AF projects
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
15
Cognitive Science applied to real-world problems
Project Nemo: Cognitive Models of Situation Assessment (currently funded by Office of Naval Research)
Nature, detection, and correction of errors in rule-based programming tasks (currently funded by National Science Foundation)
Interactive routines and cognitive workload (currently funded by Air Force Office of Scientific Research)
Project Ernestine (how cognitive modeling saved the phone company big $$$)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
16
The Précis of Project Ernestine
An Overview of a Validation of Cognitive Modeling
• Interest– Modeling: parallel performance; event-driven;
complexity in the world; transfer– Application: can modeling be used to evaluate
design; operating expenses = $3 million/sec– Theory: can cognitive theory be applied via cognitive
modeling to predict time-critical human performance in the real-world
– Validation of both modeling & the cognitive approach
• See Gray, John, & Atwood (1993) for more details
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
17
QuickTime™ and a decompressorare needed to see this picture.
Nature, detection, & correction of errors
• Topic: Can we predict “where” and “when” errors are made based upon a cognitive analysis of the procedural knowledge?
• Interest– Theory: display-based problem-solving; situated
cognition; errors; role of “verifies” in procedural skill
– Application: development of guidelines for providing feedback in “walk-up and use” devices
– Modeling: data-driven vs in-the-head cognition
• Technology: models written in ACT-R “read” from and “press” buttons on a VCR device simulation written in HyperCard
Errors and other situated pitfalls in a “walk-up and use” device
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
18
Cognitive Models of Situation Assessment
• Subjects: Submarine Action Officers• Question: Before deciding what to do, they must first
decide what is going on. How do they do this?– (Can’t open the window and look out.)– All data is passive (noises in the ocean)– Low signal-to-noise ratios are the norm
• Interest– Modeling: situation assessment aspect of decision making;
display-based problem-solving– Application: models will provide input into to the design of
displays and work groups for the New Attack Submarine– Theory: schema-theory; display-based cognition; situated
action
• ONR funded.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
19
Interactive Routines and Cognitive Workload
• Mental workload is a colloquial term that various researchers have attempted to translate into information processing terms
• Interactive routines– Elements of interactive routines are individual
productions that take from 30 to 300 msec to occur– The interactive routines themselves require 300 msec
to 3 sec to execute (ballpark estimate of time required for system to do about 10 productions)
– Requirements of one interactive routine may influence the execution of another interactive routine (that is, individual interactive routines may be considered as elements in larger strategy)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
20
Cognitive Science applied to more real-world problems
Profile before Optimizing: A Cognitive Metrics Approach to Workload Analysis
Soft constraints in Judgment & Decision-Making
Recognizing User Affective States for Active Assistance
Resolving the Paradox of the Active UserQuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
21
Paradox of the Active User
• Inefficiencies in the form of stable suboptimal performance may persist even after years of specialized experience and daily use
• These inefficiencies result in:– Increased cognitive workload– Subtle effects on performance and errors– Shifts between interaction-intensive strategies
that rely on knowledge in-the-world versus memory-intensive strategies that rely on knowledge in-the-head
• Can we predict it?• Can we eliminate it?
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
22
Profile before Optimizing: A Cognitive Metrics Approach to
Workload Analysis• Intelligence Analysts are the designated user of a
score or more projects that will put new and innovative information technologies on their desktops
• Our role is to assist the Intelligence Analysts as user of these technologies– Vital that a new type of usability assessment be
made; one that provides metrics for cognitive workload that address various cognitive factors
– Our proof-of-concept is a model that performs a task that the IAs find challenging
– From the trace of the model, we derive a cognitive metrics profile that pinpoints dynamic changes in workload demands on human cognitive, perceptual, or action systems
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
23
Soft constraints in Judgment & Decision-Making
• Is decision-making like buying a train ticket or like sailing a boat?– Is a decision about how to solve a problem
made up front and never altered until the solution is reached?
– Or does each step bring new problems and each solution new perspectives?
– How does the cost-benefit structure of the task environment influence the strategies that the decision-maker adopts?
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
24
Recognizing User Affective States for Active Assistance• Innovative research between RPI’s
Engineering Department and Cognitive Science Department
• What can happen when you combine a system that can parse emotions in real-time with . . .
• A computationally based, architecture of cognition?
• And put both in service of determining when and how to intervene to assist the human operator in a real-time, safety critical task?