1 Understanding and Improving Intelligence Analysis: Learning from other Disciplines 12-13 July, 2012 * London, England * RAF Club The purpose of this conference is to engage in a cross-disciplinary discussion about the value of learning from other fields to improve both the understanding and the practice of intelligence analysis. It will also create the network and infrastructure for an international research collaboration for the study of intelligence analysis. Intelligence, like journalism, involves the acquisition, evaluation, and dissemination of information. In 1949, Sherman Kent, described as the father of US intelligence analysis, said: “Intelligence organizations must also have many of the qualities of those of our greatest metropolitan newspapers. …They watch, report, summarize, and analyze. They have their foreign correspondents and home staff…. They have their responsibilities for completeness and accuracy—with commensurately greater penalties for omission and error. . . They even have the problem of editorial control…. Intelligence organizations (should) put more study upon newspaper organization and borrow those phases of it which they require.” But the similarities between intelligence analysis and journalism are not unique. Professionals in other fields—including medicine, the social and behavioural sciences, history and historiography, anthropology and other disciplines engaged in ethnographic research, econometric forecasting, and legal reasoning— also face many similar challenges to those that exist in intelligence analysis, including: Difficulties acquiring information from a wide variety of sources Vetting and evaluating the information that is acquired Deriving understanding and meaning from that information Impact of deadlines, editing, and other production processes on accuracy of analysis and assessment Problems in dissemination and distribution to consumers or customers Managing relationship between producer and consumer (role, responsibility, independence & objectivity) Developing professional infrastructure (recruit, select, train, & develop personnel; code of ethics) Overcoming impact of changing technology and alternative information distribution systems How do practitioners in various non-intelligence fields overcome these kinds of challenges? How are their challenges similar to or different from those that exist in the intelligence arena? What can be learned from the comparison? This event has been funded through a grant from the Brunel University Research and Innovation Fund. They are organized and hosted by Brunel University’s Centre for Intelligence and Security Studies in collaboration with University of Mississippi’s Center for Intelligence and Security Studies.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Understanding and Improving Intelligence Analysis:
Learning from other Disciplines
12-13 July, 2012 * London, England * RAF Club
The purpose of this conference is to engage in a cross-disciplinary discussion about the value of learning
from other fields to improve both the understanding and the practice of intelligence analysis. It will also
create the network and infrastructure for an international research collaboration for the study of
intelligence analysis.
Intelligence, like journalism, involves the acquisition, evaluation, and dissemination of information. In
1949, Sherman Kent, described as the father of US intelligence analysis, said:
“Intelligence organizations must also have many of the qualities of those of our greatest metropolitan
newspapers. …They watch, report, summarize, and analyze. They have their foreign correspondents and
home staff…. They have their responsibilities for completeness and accuracy—with commensurately
greater penalties for omission and error. . . They even have the problem of editorial control….
Intelligence organizations (should) put more study upon newspaper organization and borrow those
phases of it which they require.”
But the similarities between intelligence analysis and journalism are not unique. Professionals in other
fields—including medicine, the social and behavioural sciences, history and historiography, anthropology
and other disciplines engaged in ethnographic research, econometric forecasting, and legal reasoning—
also face many similar challenges to those that exist in intelligence analysis, including:
Difficulties acquiring information from a wide variety of sources
Vetting and evaluating the information that is acquired
Deriving understanding and meaning from that information
Impact of deadlines, editing, and other production processes on accuracy of analysis and
assessment
Problems in dissemination and distribution to consumers or customers
Managing relationship between producer and consumer (role, responsibility, independence &
objectivity)
Developing professional infrastructure (recruit, select, train, & develop personnel; code of ethics)
Overcoming impact of changing technology and alternative information distribution systems
How do practitioners in various non-intelligence fields overcome these kinds of challenges? How are their
challenges similar to or different from those that exist in the intelligence arena? What can be learned from
the comparison?
This event has been funded through a grant from the Brunel University Research and Innovation Fund.
They are organized and hosted by Brunel University’s Centre for Intelligence and Security Studies in
collaboration with University of Mississippi’s Center for Intelligence and Security Studies.
2
Conference agenda Day 1: Thursday 12 July
8:00 Registration and Coffee
9:30 Welcome Part 1: Stephen Marrin (Brunel University): Learning from Other Disciplines, including Medicine and
Journalism
10:00 Comparisons to Other Fields Part 1 Chair: Stephen Marrin (Brunel University)
David Chuter (Center for Security Sector Management; Cranfield University): Intelligence Analysis
as a Type of Information Processing
Michael Robinson (Deloitte and Touche, LLP): Intelligence Analysis and Digital Forensics
Carl Jensen (University of Mississippi): Intelligence Analysis and Futures Methodology
11:00 Break
11:30 Comparisons to Other Fields Part 2 Chair: Carl Jensen (University of Mississippi)
Anne Bishop (Snell & Wilmer, LLP, Phoenix AZ): Intelligence Analysis and Legal Reasoning
Anthony C. Cain (Air Force Research Institute): Intelligence Analysis and Professional Journal
Publishing
William Lawhead (University of Mississippi): Seeing Is Not Believing: Insights for Intelligence
Analysis from Professional Magicians.
12:30 Lunch
1:30 Intelligence Analysis and Social Science
Chair: Stephen Marrin (Brunel University)
Stefania Paladini (Coventry University): Bayesian analysis in intelligence and social sciences: a tool
for all trades?"
2:30 Break
3:00 Intelligence Analysis and History Chair: Philip Davies (Brunel University)
Melissa Graves (University of Mississippi): Intelligence Analysis and Historiography
Howard Clarke (The Intelligence Solutions Group) Seekers after Truth: Cross-Disciplinary Insights
for the Intelligence Profession from the Biblical and Theological Studies Discipline
WJR Gardner (Naval Historical Branch, UK Ministry of Defence): The Unreliable Memoirs of an
Applied Historian … and Operational Analyst and …
4:00 Break
4:15 Intelligence Analysis and Perception Chair: Melissa Graves (University of Mississippi)
Patricia Rooney (National Ground Intelligence Center, US Army) and Peter Ronayne (Federal
Executive Institute): Neuroscience in Intelligence Analysis: Understanding How the Brain Works and
Its Impact on Analytic Decision Making
Musa Tuzuner (Intelligence Studies Research Center; Turkish National Police Academy): Multi-
Level Cognitive Biases in the Chain of Intelligence Flow—From Source to Consumers
Mandeep Dhami (University of Surrey): Intelligence Analysis and Decision Science (Putting the
Science into the Art of Intelligence Analysis)
5:30 Day One Adjourns
3
Conference agenda Day 2: Friday 13 July
8:00 Registration and Coffee
9:30 Welcome Part 2
9:45 Evaluating Intelligence John Kringen (former CIA Director of Intelligence; Institute for Defence Analysis): Judging
Intelligence Success and Failure
10:45 Evaluating Intelligence
Chair: Philip Davies (Brunel University)
Stephen Marrin (Brunel University): Evaluating the Quality of Intelligence Analysis: By
What (Mis) Measure?
Rhiannon Gainor and France Bouthillier (McGill University): Measuring Intelligence
Success: Conceptual and Methodological Challenges of Intelligence Measurement
11:15 Break
11:30 Intelligence Analysis and the Dismal Science Chair: Julian Richards (University of Buckingham)
Michael Herman (Oxford University): Intelligence Analysis and Government Statistics
Nick Hare (UK Ministry of Defence): Intelligence Analysis and Economics
12:30 Lunch
1:30 Improving Communication Chair: Julian Richards (University of Buckingham)
Sean Newman (Fire Department of New York): How Adherence to Journalistic Standards
Positioned FDNY Watchline at the Forefront of Fire Service Analytical Intelligence
Douglas Bernhardt (South Africa: Wits Business School): Intelligence Reporting: What
Works, What Doesn't and How to Fix It
Gabriel Sebe (Bucharest University): Political Marketing Intelligence
2:30 Break
3:00 Applying Knowledge from Other Fields to Increase Understanding Peter Nardulli and Dan Roth (University of Illinois): Advancing our Understanding of Global
Patterns of Civil Unrest: An Interdisciplinary Effort
Kathleen Vogel (Cornell University): Current Intelligence Reporting and the H5N1 Bird Flu
Virus: Insights from the Field of Science and Technology Studies
4:00 Break
4:15 Practitioner views, summary, and next steps TBD
5:30pm Conference Adjourns
4
ABOUT THE SPEAKERS Douglas Bernhardt lectures in the subject of Competitive Intelligence at Wits Business School,
Johannesburg, the University of Stellenbosch Business School, and the Nelson Mandela Metropolitan
University Business School, Port Elizabeth. He is also an adjunct instructor in Competitive Intelligence
for Mercyhurst University. Previously, Douglas served as an adjunct professor of Business Intelligence &
Corporate Security at the European campus of the Thunderbird School of Global Management. He has
also taught at IMD, Lausanne and the Rotterdam School of Management. He is the author of three books,
including Competitive Intelligence: Acquiring and using corporate intelligence and counterintelligence,
published in 2003 by Financial Times Prentice-Hall. From 1993-2001 Douglas served as Managing
Director for one of Europe’s leading competitive intelligence consultancies, Business Research Group
SA, in Geneva and London, supporting the CI requirements for major firms in industries such as
pharmaceuticals, telecommunications, and FMCGs. He was a member of the Board of Directors of the
US-based Society of Competitive Intelligence Professionals (SCIP) from 1996-1999. His early
commercial experience includes the defence industry and foreign trade.
Anne W. Bishop is an attorney at Snell & Wilmer, LLP, in Phoenix, Arizona. Prior to attending law
school, Ms. Bishop worked at the National Security Agency at Ft. Meade, Maryland where she held
positions as a linguist and analyst, and taught intelligence analysis to other analysts. Following a
fellowship working for a New York Congresswoman, she worked in NSA’s legislative affairs office,
working with the House Permanent Select Committee on Intelligence Issues. Ms. Bishop’s legal practice
focuses on health care compliance, health care litigation, and business disputes. She advises physicians
and hospitals on various health-care related issues. Ms. Bishop is also a member of the privacy and data
security group. Ms. Bishop is a frequent writer and speaker on health care compliance and legal ethics.
Ms. Bishop graduated in 1986 from the University of Pennsylvania with a degree in international
relations & in 2006 from the Sandra Day O’Connor College of Law at Arizona State University.
Dr. Anthony C. (Chris) Cain is Chief of Academic Affairs at The Air University, Maxwell Air Force
Base, Alabama. He received his M.A. and Ph.D. in Military History from The Ohio State University, and
also holds Masters Degrees from the Air War College and Embry-Riddle Aeronautical University, as well
as a B.S. from Georgia State University. Dr. Cain is a graduate of the Air Command and Staff College
(ACSC), and the Air War College. Prior to assuming his present duties, he served as the Deputy Director
of the Air Force Research Institute (AFRI), Dean of AFRI, Chief of AFRI’s Outreach Division, founding
Editor-in-Chief of Strategic Studies Quarterly, Research Director and Dean of Education & Curriculum at
ACSC, and as the Chief, Professional Journals Division & editor of the Air and Space Power Journal at
the College of Aerospace Doctrine, Research and Education. Dr. Cain is the author of The Forgotten Air
Force: The French Air Force and Air Doctrine in the 1930s (Smithsonian Institute Press, 2002), as well
as book chapters and articles on issues dealing with airpower, professional military education, effects-
based operations, regional threats from weapons of mass destruction, & military history.
Dr. David Chuter worked for more than thirty years for the UK government in the defence and security
area. He was involved, among other subjects, in the negotiations leading to the Maastricht Political Union
Treaty, and subsequent discussions about a European security and defence policy, as well as arms control,
weapons of mass destruction, and war crimes and transitional justice. His last job was as Special Adviser
to the Policy Director of the French Ministry of Defence in Paris. He took early retirement at the end of
2008 and is now an author, lecturer, translator and consultant based in Paris. Dr Chuter is a lecturer at
Sciences Po in Paris, where he teaches courses in Intelligence and Policy-making, and in Crisis
Management. He is also Senior Research Fellow at the Centre for Security Sector Management at
Cranfield University, on whose behalf he teaches security-related subjects in various countries, and Chair
of the Governing Board of the International Security Information Service Europe. He is the author of four
books, and many articles, most recently Governing and Managing the Defence Sector (2011). His current
research interest is the politics and practical problems of the Rule of Law in the Security Sector.
5
Howard Clarke, MCS, CCA is an internationally experienced law enforcement intelligence practitioner.
During his thirty-year law enforcement career Howard held senior intelligence analysis positions with law
enforcement agencies in Australia and Canada. For the past decade he has specialized in strategic
intelligence analysis applications in law enforcement and homeland security and he remains active in
intelligence analysis training and consulting work. He has taught extensively on strategic intelligence
analysis and open source intelligence issues in the U.K., Europe, Asia and North America and is a
published author on Insider Threat issues. He holds a Master of Christian Studies from Regent College
International Graduate Theological School. Howard is currently an intelligence analysis instructor at the
British Columbia Institute of Technology and the Justice Institute of British Columbia in Canada and he
formerly held adjunct faculty status with Henley-Putnam University in the U.S. He is a Lifetime Certified
Criminal Analyst, a former member of the Board of Governors of the Society of Certified Criminal
Analysts (SCCA), a member of the International Association of Law Enforcement Intelligence Analysts
(IALEIA) and the International Association for Intelligence Education (IAFIE).
Dr. Philip H.J. Davies is a political sociologist specialising in the study of national intelligence
institutions. He is Director of the Brunel Centre for Intelligence and Security Studies (BCISS). In 2010
BCISS was awarded University Interdisciplinary Research Centre status and now includes scholars from
across the University from departments and schools as diverse as Engineering and Design, Economics
and Law in addition to its core team based in Politics and History. During 2009-10, Dr. Davies headed
BCISS work on the new UK military Joint Intelligence Doctrine. On this project, Brunel was selected by
the Ministry of Defence as a full partner with its Development, Concepts and Doctrine Centre and
Defence Intelligence (former the Defence Intelligence Staff). This work has involved both the
fundamental rethinking and reformulation of intelligence doctrine (JDP 2-00 Understanding and
Intelligence in the Joint Operating Environment) but also the articulation of an entirely new doctrine on
‘understanding’ (JDP 04 Understanding). Before that Dr. Davies ran an ESRC-funded international
seminar series entitled ‘Intelligence and Government in the 21st Century’ working in collaboration with
Dr. Robert Dover at Loughborough. He has also recently completed a major and intensive comparative
study of British and American national intelligence has been published by Praeger Security International
in 2012. In 2004 Dr. Davies designed and, until 2010, convened Brunel’s highly successful MA in
Intelligence and Security Studies, including its ground-breaking Brunel Analytical Simulation Exercise.
BASE is term-long practical in strategic intelligence assessment in which students are divided into
drafting teams modelled on the UK Joint Intelligence Organisation analytic staff and produce JIO-style
assessments on live, real-world topics using open sources.
Dr. Mandeep K. Dhami is a Reader in Forensic Psychology at the University of Surrey. Previously she
held academic appointments at the University of Cambridge (UK), University of Victoria (Canada), the
University of Maryland (USA), and the Max Planck Institute for Human Development (Germany). She
has worked as a Senior Scientist for the Defence Science and Technology Laboratory (MoD). Dr. Dhami
has a PhD in Psychology and an MA in Criminology. Her research draws conceptual connections between
Psychology (esp. decision science and social cognition) and Criminology. She has examined topics such
as decision making, risk, and forecasting. She has applied these issues to the criminal justice system, and
more recently to the defence and security sectors (with a focus on intelligence analysis and cyber issues).
To-date, she has over 65 publications, and is lead editor of a book entitled “Judgment and Decision
Making as a Skill: Learning, Development and Evolution” published by Cambridge University Press in
2011. Her research has won several awards including from the European Association of Decision Making
and the Society for the Psychological Study of Social Issue (SPSSI, Division 9 of the American
Psychological Society). Dr. Dhami is Fellow of SPSSI (Division 9 of the American Psychological
Association), and is on the editorial board of several journals including Perspectives on Psychological
Science. She has provided advice and consultancy to various government bodies including the Home
Office, Ministry of Justice, Government Communications Headquarters, and the Metropolitan Police
(New Scotland Yard).
6
Rhiannon Gainor is a PhD candidate at McGill University’s School of Information Studies, a
McConnell Foundation Fellow in 2010/2011, and a recipient of Québec’s Fonds de recherche Société et
culture (FQRSC) grant. Her research interests are knowledge management, competitive intelligence, and
information metrics. She has a Master’s of Library and Information Studies, and Master’s of Arts in
Humanities Computing from the University of Alberta.
W J R (Jock) Gardner served in the Royal Navy for 30 years, specialising in anti-submarine warfare.
He served on operational, analysis and intelligence staffs. In 1989 he was awarded an M Phil in
International Relations at the University of Cambridge, writing a dissertation on Soviet nuclear ballistic
missile submarines. He was also the Editor of The Cambridge Review of International Affairs. Leaving
the Navy in 1994, he joined the Naval Historical Branch as a Historian and taking an especial interest in
World War II, antisubmarine warfare and intelligence. His publications include Anti-submarine Warfare
(Brassey’s, 1996) and Decoding History: the Battle of the Atlantic and Ultra (Naval Institute Press,
1999). He has lectured on naval historical and intelligence topics from Moscow to Monterey. He is the
external examiner at Brunel University for the MA in Intelligence and Security Studies, a member of the
Editorial Board of The Mariner’s Mirror and a Fellow of the Royal Historical Society.
Melissa Graves, JD, MA serves as Project Coordinator and Instructor at the University of Mississippi’s
Center for Intelligence and Security Studies. Among her accomplishments, she and her co-developer
Walter Flaschka have designed and implemented the Days of Intrigue, a realistic practical exercise that is
conducted yearly at UM and which involves numerous intelligence community agencies. Ms. Graves
received her BA with a double major in English and Communications from Hardin-Simmons College, an
MA in History from UM, and a JD from the UM School of Law. She is presently pursuing a Ph.D. in
History. Her work will appear in upcoming publications, including Introduction to Intelligence Studies
(Carl Jensen, David McElreath, and Melissa Graves) and Intelligence Studies in Britain and the US:
Historiography since 1945 (ed. Christopher Moran).
Nick Hare is a professional economist, and has worked in a number of roles in the UK Ministry of
Defence (MOD) since graduating in 1999 with an MA in Philosophy and Economics from the University
of Edinburgh. He is currently head of the Futures and Analytical Methods (FAM) team in the MOD. The
FAM team works to promote the aims of the Professional Head of Defence Intelligence Analysis. This
involves encouraging analysts to use structured and auditable methods to refine intelligence requirements,
identify assumptions, generate hypotheses and scenarios, test hypotheses using data, collect information,
and describe conclusions using appropriate probabilistic language. The FAM team also researches new
and developing tools and approaches that are of potential use to intelligence analysts, and works with
other departments to promote professionalisation of analysis across government.
Michael Herman served from 1952 to 1987 in Britain's Government Communications Headquarters,
with secondments to the Cabinet Office (as Secretary of the Joint Intelligence Committee) and to the
Ministry of Defence.. His Intelligence Power in Peace and War was published in 1996 and has been
regularly reprinted. His Intelligence Services in the Information Age was published in 2001, and he has
co-edited and contributed to Intelligence in the Cold War: What Difference Did It Make (in publication).
He is currently an Honorary Departmental Fellow at Aberystwyth University and an Associate Member of
Nuffield College Oxford, as Founder Director of the Oxford Intelligence Group. He is an Honorary D.Litt
of Nottingham University.
7
Dr. Carl J. Jensen III. is a 1978 graduate of the U. S. Naval Academy. He served in the Navy from 1978
until 1983, first aboard the nuclear fleet ballistic missile submarine USS George Washington Carver and
then as an aide to the Commander of Submarine Group Five. Dr. Jensen graduated from FBI New Agents
Training in 1984 and served as a field agent in Atlanta, Georgia, Monterey, California, and Youngstown,
Ohio. In August 1992, Dr. Jensen reported to the FBI Laboratory where he received certification as a
Racketeering Records Examiner. In June 1997, Dr. Jensen reported to the Behavioral Science Unit at the
FBI Academy in Quantico, Virginia, where he instructed senior police officials at the FBI National
Academy, conducted research, provided consultation, and served as Assistant Unit Chief. Upon his
retirement from the FBI in 2006, Dr. Jensen joined the RAND Corporation as a Senior Behavioral
Scientist. In 2007, he joined the Legal Studies faculty at the University of Mississippi, where he currently
serves as director of the Center for Intelligence and Security Studies. He is the 2012 recipient of the
Outstanding Instructor of the Year Award from the International Association for Intelligence Education, a
2008 recipient of the Thomas Crowe Outstanding Faculty Award from the University of Mississippi and a
2004 recipient of the Jefferson Award for Outstanding Research from the University of Virginia. Dr.
Jensen holds a Master of Arts degree from Kent State University and a Doctor of Philosophy degree from
the University of Maryland. He has instructed throughout the world and is the author of numerous
articles, books, book chapters and technical reports.
Dr. John A. Kringen is a Research Staff Member with the Institute for Defense Analyses in Alexandria
Virginia. He recently retired from the Central Intelligence Agency after more than three decades of
government service. During 2005-2008 he served as the Director of Intelligence at CIA where he
managed and led the activities of several thousand CIA analysts and staff. Prior to becoming Director of
Intelligence, he served in a variety of managerial assignments in that Directorate, including as Director of
the CIA’s Crime and Narcotics Center. Beyond CIA, he served twice overseas in Europe, most recently
as a senior intelligence advisor to United States European Command from July 2008 to August 2011. In
the late 1990s, he was head of imagery analysis at the predecessor organization to the National Geospatial
Intelligence Agency. He received a Ph.D. from the University of Minnesota in 1978.
Dr. William Lawhead retired from the University of Mississippi in summer 2012 after having completed
seven years as Chair of the Department of Philosophy and Religion. He also completed forty-one years of
teaching philosophy, thirty-two of them at UM. Among the courses he has taught are history of
philosophy, logic and critical thinking, and philosophy of science. He has served on the advisory board
of the University of Mississippi’s Center for Intelligence and Security Studies and has participated in and
given papers at 5-Eyes Conferences. He is a member of the International Association for Intelligence
Education. He has published two books. These are a history of Western philosophy, The Voyage of
Discovery, 3rd
ed. (Wadsworth, 2007) and an introduction to philosophy, The Philosophical Journey, 5th
ed. (McGraw-Hill, 2011). He is working on a third book, Taking Philosophy Seriously, for Oxford
University Press. Dr. Lawhead received his Ph.D. in philosophy from the University of Texas, Austin.
Dr. Stephen Marrin is a Lecturer in the Department of Politics and History at Brunel University in
London, England. He received his PhD and MA in foreign affairs from the Department of Politics at the
University of Virginia. He has written about many different aspects of intelligence analysis, including
new analyst training at CIA’s Sherman Kent School, the similarities and differences between intelligence
analysis and medical diagnosis as a source of ideas for improving the quality of future intelligence
analysis, and the professionalization of intelligence analysis. In 2004 the National Journal profiled him as
one of the ten leading experts on intelligence reform. He previously worked as an analyst at both the US
Central Intelligence Agency and the US Government Accountability Office, and taught in the Intelligence
Studies Department at Mercyhurst University.
8
Dr. Peter F. Nardulli is Professor of Political Science and Law at the University of Illinois at Urbana-
Champaign, the founding Director of the Cline Center for Democracy, and the editor of a book series with
the University of Illinois Press: Democracy, Free Enterprise and the Rule of Law. He has been on the
faculty at UIUC since 1974 and served as department head in Political Science from 1992 until 2006.
Nardulli is the author of six books on various aspects of the legal process and empirical democratic
theory. He has authored a number of articles in journals such at the American Political Science Review,
Public Choice, Political Communication, Political Behavior and a number of law reviews. Nardulli is
currently directing a global study, the Societal Infrastructures and Development Project (SID). SID uses a
number of technologically advanced, innovative methodologies to examine the impact of political, legal
and economic institutions on a wide range of societal development indicators (economic growth, human
Pritchard, Matthew C. & Michael S. Goodman (2008): Intelligence: The Loss of
Innocence, International Journal of Intelligence and CounterIntelligence, 22/1, 147-164.
(ARCHAEOLOGY)
Rieber, Steven, and Neil Thomason. "Toward Improving Intelligence Analysis: Creation of a National
Institute for Analytic Methods." Studies in Intelligence 49, no. 4 (2005): 71-77. (MEDICINE)
Rodgers, R. Scott. "Improving Analysis: Dealing with Information Processing Errors." International
Journal of Intelligence and Counterintelligence 19/4 (Winter 2006-2007): 622-641. (PSYCHOLOGY)
Spivey, Robin V. "The Devil Is in the Details: The Legal Profession as a Model for Authentic Dissent."
International Journal of Intelligence and Counterintelligence 22/4 (Winter 2009): 632-651.
(LAW/LEGAL REASONING)
Weiss, Charles. "Communicating Uncertainty in Intelligence and Other Professions." International
Journal of Intelligence and Counterintelligence 21, no 1 (Spring 2008): 57-85.
(GENERAL/SCIENCE/LAW)
Young, David. Scholarship, Intelligence, and Journalism. Reuters Fellows Lecture. March 2004.
(JOURNALISM)
13
Paper and Presentation Abstracts Understanding and Improving Intelligence Analysis:
Learning from other Disciplines
12-13 July, 2012 * London, England * RAF Club
Douglas Bernhardt (South Africa: Wits Business School): Intelligence Reporting: What
Works, What Doesn’t and How to Fix It: This paper argues that however whatever the quality of the
findings and analysis of an intelligence deliverable, it will often fail to achieve its main objective; that is, to provide
a decision-maker with product that he or she finds both compelling and relevant to his or her specific decision
challenges. This occurs mainly for two reasons: (1) the ‘disconnect’ which often exists between intelligence
consumers and producers and (2) the ‘packaging’ of finished intelligence as a set of dry, colourless news and facts.
This author, a long-time practitioner and lecturer in the field of Competitive Intelligence, has previously described
the phenomenon of the sub-optimal relationships which exist in many firms between Competitive Intelligence units
and their executive customers. In this paper he not only revisits and updates earlier discussions, but adds a fresh
dimension; one which centres around the process of persuasion. In short, if an intelligence product is not persuasive
(think the PDB of 6 August 2001), it has no value; it’s just another report to digest. This paper will consider the
factors which can and should make an intelligence briefing, or report, persuasive; borrowing, in part, from the fields
of advertising and sales. It will also focus on the unique contribution that human source information (HUMINT)
makes to intelligence analysis and reporting OSINT is important, but it’s not enough. The paper/presentation we
propose will explore two principle topics: (1) what are the issues. involved in, and what needs to be done to
overcome, the challenges inherent in the producer-consumer relationship? and (2) what tools can we apply to
improve, and sometimes totally recast, the packaging and delivery of intelligence product that makes a difference,
that serves as a springboard for its users to gain ‘competitive advantage’? Although this presentation will be
discussed within a corporate sector context, the key lessons are equally relevant to the domains of national security
and law enforcement.
Anne Bishop (Snell & Wilmer, LLP, Phoenix AZ): Intelligence Analysis and Legal
Reasoning: This presentation will explore the similarities between legal reasoning and intelligence analysis. The
presenter is a practicing attorney who spent about 15 years in the U.S. intelligence community, working as an
analyst and linguist and training intelligence analysts. Ms. Bishop will focus on the similarities in acquiring relevant
information, evaluating sources, and deriving meaning from that information. In both intelligence analysis and legal
research, locating the relevant information is critically important. Ms. Bishop will discuss access, search strategies,
and ways to validate search results. In addition, Ms. Bishop will discuss the relatively static hierarchy of sources
that exists in the legal research realm. However, the bulk of the presentation will focus on analytic techniques that
are central to legal analysis-- analogies, extrapolation, legal reasoning, and the importance of context. Lawyers are
taught in law school to draw analogies between their current cases and existing case law precedent. By creating a
compelling analogy, a lawyer is likely to prevail in a court. Similarly, by drawing analogies to prior events, an
intelligence analyst can analogize what is transpiring today based on similar prior events. In addition, Ms. Bishop
will discuss extrapolation and its value to both intelligence analysis and legal reasoning. That is, in legal reasoning,
there is often no legal principal that is completely similar. Lawyers learn to find a case that is analytically similar
and extrapolate the principles to the situation at hand. Similarly, intelligence analysts can look at an existing set of
facts and extrapolate from known facts into the unknown. In legal reasoning, lawyers look at two existing cases and
draw inferences regarding how those two cases fit together in order to determine how a court should rue on a current
case. This technique could be used predicatively by intelligence analysts. Finally, Ms. Bishop will discuss the
importance of context. Context is key in determining a legal outcome. The same is true for intelligence analysis.
Determining the status quo is critical in both context in order to figure out the potential outcomes.
14
Anthony C. Cain (Air Force Research Institute): Intelligence Analysis and Professional
Journal Publishing: Publishing professional military journals requires many of the same processes and
decision capabilities found in intelligence analysis. Professional journals, by their nature, can narrowly
focus on technical or tactical subjects, but the selection and editing of content requires broad knowledge
in several areas. For example, journals that focus on operational issues may receive article submissions
that deal with the full spectrum of service or joint capabilities. Journal editors will rarely be trained or
equipped to evaluate the credibility and merits of such a broad spectrum of content. Therefore, creating
mechanisms to ensure the credibility and quality of journal content form the essential tasks for
professional journal editing. Those same mechanisms could be adapted to serve intelligence analysis
processes. Specific cases from the author’s experience as the editor of Air and Space Power Journal and
as founding editor-in-chief of Strategic Studies Quarterly will illustrate the challenge and solutions
adopted to preserve the professional credibility of both journals.
David Chuter (Center for Security Sector Management; Cranfield University): Intelligence
Analysis as a Type of Information Processing Although intelligence analysis is obviously a special case of the collection, processing and use of
information, it is not entirely unique. Indeed, we carry out analogous tasks to intelligence analysis all the
time, not only professionally, but in our personal lives as well. Intelligence analysis can be seen as a type
of information processing which has a special combination of characteristics, generally not found together
elsewhere. We can analyse all information processing under three types of heading. The first is the degree
of applicability, which is to say how far the process is designed to lead to an operational outcome. At one
extreme, practitioners of emergency medicine have a short period in which to gather, process and act on
information in order to save a life. At the other extreme, scientists, archaeologists or sociologists may
spend years on a research programme which simply validates prior assumptions, and does not have any
operational output at all. Intelligence is in principle only collected in response to specific tasking to
answer specific questions, and so in principle is a high-applicability process. The second is the degree of
completeness of the information which can reasonably be desired. Surgeons contemplating major, non-
acute surgery, will exhaustively research everything they can, to make sure that what they do is as
effective as possible. At the other extreme, many decisions are successfully taken in everyday life on the
basis of very incomplete information: if you arrive at an airport and realize that you have left your adaptor
plug at home, you will decide to buy the first thing you see that functions correctly. There are many
intermediate cases: historical research, for example, can never in principle be complete or definitive, no
matter how long it continues. Criminal trials generally only consider enough evidence to establish guilt
beyond a reasonable doubt, and some potential crimes may not even be charged. Intelligence is by
definition always incomplete, and its reliability is always suspect. The third is the need for and
expectation of accuracy. Most medical decisions have to be very accurate, since otherwise the patient will
be harmed, and there are professional sanctions for those who make mistakes. At the other extreme,
journalism and the productions of advocacy NGOs are seldom expected to be fully accurate, and there are
few professional sanctions for inaccuracy or even downright untruths. There are intermediate cases such
as investment advice, where there is an expectation of at least broad accuracy, but few sanctions in its
absence. There are also cases (like the conclusions of juries in criminal trials) where the degree of
accuracy is not knowable, and all one can ask is whether the verdict is reasonable in the circumstances.
Intelligence is generally an area with high (and often excessive) expectations of accuracy, and a great
requirement for it. The special combination of high applicability, lack of completeness and significant
demand for accuracy makes intelligence analysis a special and interesting case.
15
Howard Clarke (The Intelligence Solutions Group) Seekers after Truth: Cross-Disciplinary
Insights for the Intelligence Profession from the Biblical and Theological Studies Discipline Biblical hermeneutics, or exegesis, is an interpretative activity that contains some significant parallels
with important facets of the intelligence analysis endeavor. The term exegesis refers to careful
investigation of the original meaning of texts in their historical and literary contexts. The process involves
asking analytical questions about various aspects of the source materials and their contexts, and it implies
careful, detailed analysis. As an analytic discipline, effective biblical interpretation requires a capacity to
deal with challenging data sets, complex source evaluation circumstances, and it seeks to provide insights
which will shape and influence the beliefs and actions of a significant community (i.e. the consumers). In
principle, a process not dissimilar to the Intelligence Cycle is employed and at various times and in
various contexts there is a need to utilize techniques which can be construed as forms of HUMINT,
OSINT, and TECHINT. As a result of the significant analytic challenge involved a growing body of
interpretative techniques and methodologies has developed over many centuries. In the sense that
intelligence analysis is essentially concerned with the search for truth and meaning (albeit in particular
geopolitical, security and various specific contexts) and the communication of resultant insights to
decision-makers, then there are correlations to be drawn with the sub-disciplines of biblical exegesis and
hermeneutics and the overarching theological discipline they inform. Particular challenges that must be
addressed in biblical interpretation include: (1) the challenge of distance of time – the key events and
communications to be understood are separated from the contemporary interpreter by a vast gulf of time;
(2) the challenge of cultural distance – key actors and events reside within an ancient agrarian, middle
eastern context; (3) the challenge of geographical distance – requires an appreciation of the geography of
the Middle East at various points in history; (4) the challenge of linguistic distance – communication
occurs in the languages of Hebrew, Aramaic and Greek in the form those languages took thousands of
years ago; and (5) • various challenges around authorship and sourcing. The successful exegetical
practitioner, like the proficient intelligence analyst, must: (a) • understand the world behind the
communication being researched – the context of the source and the subject the source is reporting on; (b)
Understand the world within that communication – the language and structuring of the communication,
what is the source saying and how is the source saying it, and what meaning does this convey?; and (c)
understand the world before the communication – the context of the message recipient (the intelligence
consumer if you like). This paper will explore and discuss various areas of similarity and dissimilarity
between intelligence analysis practice and exegetical practice and will suggest some opportunities for
productive crossdisciplinary learning for intelligence practitioners.
Mandeep Dhami (University of Surrey): Intelligence Analysis and Decision Science
(Putting the Science into the Art of Intelligence Analysis) What is the probability that Iran will have nuclear weapons in 2012? What is the risk posed by
Anonymous to UK cyber space? Which al-Qaeda operatives should be prioritized for further intelligence
gathering? These are some of the types of questions that intelligence analysts must consider. The field of
Decision Science has existed for over 60 years, and is devoted to understanding how people perform
some of the sorts of tasks that intelligence analysts are faced with. For example, decision scientists ask:
How do people judge probability, perceive risk, and make choices? How do people search and select
relevant information, and use (weight and integrate) it appropriately when making decisions? Decision
scientists with backgrounds in disciplines such as Psychology and fields such as Behavioral Economics
develop and test correspondence- or coherence-based theories. They employ quantitative methods. And,
they consider normative, descriptive and prescriptive questions. I shall discuss how past research from the
field of Decision Science can be used to shed some light on how intelligence analysts might perform as
well as how their performance can be enhanced. In addition, I explore how theories and methods from
this field can be usefully applied to understanding the art of intelligence analysis.
16
Rhiannon Gainor and France Bouthillier (McGill University): Measuring Intelligence
Success: Conceptual and Methodological Challenges of Intelligence Measurement The purpose of intelligence practices is to source and develop intelligence to inform a decision. While there is a
significant body of literature dealing on how to implement and design intelligence processes, little is known
regarding how intelligence makes a difference in organizations. Measurement allows organizations to identify
deviations in standards, benchmark success, and whether objectives are met. When an expert, equipped with
intelligence, advises a decision-maker, it can be difficult to measure not only the role the intelligence played in the
decision, but the success of the decision. While intelligence processes can be and are measured with some effort,
identification and valuation of intelligence outcomes and impacts can be elusive. The fields of
competitive/business/strategic intelligence, intellectual capital, and knowledge management struggle with the
challenge of how to measure what is intangible, subjective, and frequently delayed in manifestation, as does the
larger intelligence field. Many organizations measure process, inputs, and satisfaction, and sidestep the considerable
methodological challenges related to accurate measurement of intelligence outcomes and impacts. The literature of
intangibles measurement provides some insights applicable to intelligence measurement, such as identifying
standard practice, problems with conceptualization and current measurement models, and best practices in
developing measures. The purpose of this paper is first, to clarify terms and concepts related to measurement,
asking: what is measurement, why measurement is needed, and what are the characteristics of ‘good’ measurement,
all within the context of measuring intangible outcomes. The second purpose of this paper is to identify from the
literature of intangibles measurement best practices and frameworks for developing measures of intelligence success
that account not only for costs and processes, but also outcomes and impacts. The intent is to identify conceptual and
methodological issues related to measurement and to offer for consideration insights from various fields that should
inform measurement strategies in intelligence.
WJR Gardner (Naval Historical Branch, UK Ministry of Defence): The Unreliable
Memoirs of an Applied Historian … and Operational Analyst and … The sound practices of history and intelligence have much in common and, it can be argued, each discipline can
learn from the other. But there may, in fact, be an even larger range of disciplines which may not be quite so
obvious or indeed make quite such a clear contribution to proper intelligence method and analysis. Arguably these
might range from art history to zoology; from paleontology to psychology. In the relatively restricted compass of a
conference paper or presentation it is not possible to produce a full description far less an evaluation of all these
potential relationships. However, this paper will attempt to take some consideration of a number of other disciplines
whilst concentrating on the symbiotic relationships that exist between intelligence and history, especially as it has
related to the experience of one historian and intelligence officer. These would range from consideration of sources,
analytical methods, understanding the significance of derived reporting and – perhaps as importantly - its limitations
and, lastly, the essential skills needed in growing and maintaining an effective relationship between producer and
consumer. It will be demonstrated that not only are there theoretical advantages to be gained on both sides but there
is also a considerable track record of practitioners, moving relatively seamlessly and successfully from one
discipline to the other, and also how many of the theoretical and analytical practices of one field can be applied to
the other.
Melissa Graves (University of Mississippi): Intelligence Analysis and Historiography Historians function similar to intelligence analysts. They consume huge amounts of disparate and oftentimes
questionably reliable material. Both must piece together incomplete stories. With both intelligence and history, a
source can make or break one’s ability to compose good analysis. Historians and analysts both construct their
particular version of world based upon primary records. Likewise, they face inherent perils of analytic biases,
particularly mirror imaging. Both understand that events and people do not exist within a vacuum and they must
accurately and fully take into account many competing forces. This presentation will discuss how the use of
historical methodology can assist analysts in looking at the broader picture and understanding trends and
motivations. In each story they construct, historians examine how race, class, gender, and religion affect key
players’ decisions. By looking at historical methodology, analysts can glean tactics helpful to their own work.
17
Nick Hare (UK Ministry of Defence): Intelligence Analysis and Economics At first glance, the overlap between 'economics' and 'intelligence analysis' should be significant. Both
disciplines are concerned with using evidence to test hypotheses about behaviour, and with modelling of
the actions of individuals and systems through examination of incentives at both micro- and macroscopic
levels. Why, then, is there relatively little practical interaction and overlap between the two? Based on an
informal survey of the few individuals who, in the UK, have worked in the intersection, I will present
some key similarities and differences between the professions of economics and intelligence analysis. I
will propose some tentative hypotheses to explain these similarities and differences, and identify some
key insights that each has to offer the other.
Michael Herman (Oxford University): Intelligence and Government Statistics British intelligence and statistics have evolved as government institutions over a similar periods as part of
the ‘knowledgeable state’, and useful comparisons can be drawn between their institutions. Despite being
pulled between centralization and decentralization, both have moved towards systems with central output
and influence on standards; but the moves are uneven. Statistics now has a national statistician and
supervisory authority overseeing what is still a mixture of central and departmental units: by contrast
intelligence’s professional authority has remained its central committee system. Arguably statistics has
moved in a natural direction for government’s knowledge producers, while British intelligence retains the
committee system formed as part of the tri-service military structure with which it fought the Second War
and moved into peace.
Carl Jensen (University of Mississippi): Intelligence Analysis and Futures Methodology The National Commission on Terrorist Attacks Upon the United States (also known as the 9-11
Commission) concluded that the U.S. intelligence community exhibited a “lack of imagination” in failing
to prevent the terrorist attacks of 2001. Like intelligence analysts, futurists are concerned with forecasting
future events; likewise, they must exhibit considerable imagination in divining emerging or distant trends.
Over the years, futures research has produced strategies for improving creative thinking. As well,
professional futurists have had to learn how to adapt their creative forecasts so that they have utility for
concrete and pragmatic customers. This presentation will discuss how the methods of futurists can be
integrated into the intelligence world, both in terms of analysis and establishing and promoting dialogue.
John Kringen (former CIA Director of Intelligence; Institute for Defence Analysis):
Judging Intelligence Success and Failure
There has been little public dialogue about the appropriate standards for evaluating the performance of
intelligence organizations. In the United States, for example, intelligence performance has typically been
judged on a case-by-case basis in the wake of perceived intelligence “failures” -- with limited discussion
or debate on the standards being applied. What are realistic standards for evaluating the performance of
intelligence organizations in an uncertain world? At a minimum, they need to go beyond the notions of
surprise and embarrassment. What responsibilities do customers and those entities responsible for
oversight of intelligence have for the performance of intelligence organizations? In this regard, clear
guidance on such matters as intelligence priorities is critical. Finally, what insights on appropriate
performance standards can be gained from examining practices outside the intelligence enterprise? For
example, if major defense contractors in Europe and the United States are often challenged in terms of
their ability to project their own delivery schedules, what standards should be applied to weapons analysts
in the intelligence community? This PowerPoint presentation will propose a framework for evaluating the
performance of intelligence organizations – drawing upon debates in recent decades about “intelligence
failure” and relevant practice in other professions
18
William Lawhead (University of Mississippi): Seeing Is Not Believing: Insights for Intelligence Analysis from
Professional Magicians. To relate magic (conjuring) and intelligence analysis seems like quite a stretch. However,
throughout history magicians have served their country by using their professional skills in warfare and espionage. I
briefly mention some examples of how magicians have used their knowledge to make contributions to these two activities.
But these two activities are not the same as analysis. What do the skills and knowledge of professional magicians have to
offer to intelligence analysts? This paper will suggest that they have a lot to teach us in the areas of deception, perception,
and cognition. Deception The most obvious application of magic to intelligence analysis is in the area of deception and
counterdeception. Here, the goals of the magician and the analyst are opposite. The magician tries to deceive us and the
analyst tries to avoid being deceived or to uncover attempts at deception. But, understanding the magician’s techniques
and principles will be useful in understanding deception in the intelligence arena. Barton Whaley is a political scientist
who is a leading authority and author on the topic of military deception. He is also a magician and author of several books
on magic. Some of his insights, based on knowledge of magic, will be discussed. Perception A greater knowledge of how
we experience the world will be useful to the analyst. Here, magicians can instruct us. After all, magicians are able to
make us think we see things that aren’t there and make us not see things that are there. Recently, neuroscientists have
started doing research on why magicians fool us. For example, scientists have used eye-tracking technology on subjects
while they watched a magician perform in order to see what causes the illusion. They found that the magician’s use of
misdirection did not divert the gaze of the audience but diverted their attention. This is related to the phenomenon of
“inattentional blindness.” We do not see objects in our visual field (including the magician’s sneaky moves) even though
photons from the object are reaching our retina, because of a manipulation of our attention. Certainly, for the analyst,
staring at lots and lots of data is not enough. We have to be concerned with how the data is being viewed and how our
attention is directed or misdirected. Cognition Related to the previous topics is the issue of how the magician controls our
minds. While some people think that misdirection is the main tool of the magician, other theorists think it is the way that
the magician plants assumptions in our minds or exploits the assumptions we already have. Magician Roberto Giobbi says
“Magic should be very easy, since our spectators fool themselves. All you need to do is avoid any words, thoughts or
actions that interrupt this tendency.” Since assumption checking is an important technique for the analyst, an examination
of how magicians use our assumptions will be instructive.
Stephen Marrin (Brunel University): Learning from Other Disciplines, including Medicine
and Journalism Revisits and extends call for research agenda articulated at the end of the 2005 article “Improving Intelligence
Analysis by Looking to the Medical Profession”: Crossing Professional Lines: Finally, the lessons that intelligence
can draw from an examination of the similarities and differences with the medical profession indicate the
importance of looking to analogous professions for ideas that can be adapted to an intelligence context. Doing so
might help improve finished intelligence production processes and the incorporation of intelligence into
decisionmaking. Analogies serve a number of purposes, such as aiding communication about difficult topics by
finding illustrative examples in other fields, or by more directly affecting existing ways of doing business through
the incorporation of tools that exist to achieve similar purposes in other fields. Many of the challenges intelligence
analysts face are not as unique as its practitioners believe, but the insularity of the field prevents them from being
able to identify the lessons from other professions that could be useful as models to follow. As a result, the first task
is to identify analogous professions, and examine them for the lessons they might provide. Any profession that
encounters similar problems-such as medicine, journalism, law, or law enforcement- may provide fertile ground for
deriving ideas to improve existing practices. Perhaps if intelligence analysts adopted methods from analogous
professions-or adapted them to the unique requirements of intelligence analysis-some of the obstacles they currently
face in accurately portraying their understandings of the international environment could be overcome.” (Source:
Stephen Marrin and Jonathan Clemente. Improving Intelligence Analysis by Looking to the Medical Profession.
International Journal of Intelligence and CounterIntelligence, 18:(707-729). 2005 (pps 726-727).
19
Stephen Marrin (Brunel University): Evaluating the Quality of Intelligence Analysis: By
What (Mis) Measure?: Each of the criteria most frequently used to evaluate the quality of intelligence
analysis has limitations and problems. When accuracy and surprise are employed as absolute standards, their use
reflects unrealistic expectations of perfection and omniscience. Scholars have adjusted by exploring the use of a
relative standard consisting of the ratio of success to failure, most frequently illustrated using the batting average
analogy from baseball. Unfortunately even this relative standard is flawed in that there is no way to determine
either what the batting average is or should be. Finally, a standard based on the decisionmakers’ perspective is
sometimes used to evaluate the analytic product’s relevance and utility. But this metric, too, has significant
limitations. In the end, there is no consensus as to which is the best criteria to use in evaluating analytic quality,
reflecting the lack of consensus as to what the actual purpose of intelligence analysis is or should be.
Peter Nardulli and Dan Roth (University of Illinois): Advancing our Understanding of
Global Patterns of Civil Unrest: An Interdisciplinary Effort This paper deals with an area of increasing concern to intelligence analysis: the study of civil unrest, including
destabilizing acts initiated by private citizens and disruptive state actions. It is based on one component (the Societal
Stability Protocol, or SSP) of a six-year multidisciplinary effort, the Social, Political and Economic Event Database
project (SPEED), which was initiated by and housed in the Cline Center for Democracy. The paper will outline how
SPEED has used diverse technologies to enhance human capacities to conduct core intelligence functions: the
detection, documentation, summarization and analysis of destabilizing developments. The paper will consist of
three parts. The first will briefly outline: (1) the creation of SPEED’s global news archive, which contains over 150
million digitized news reports from a diverse set of news sources for every country in the world for the post-1945
era; (2) the creation of its destabilizing event ontology, which serves to focus the project’s detection efforts; and (3)
its use of automatic text classification and event extraction technologies to capture pertinent information within the
news archive. The second main section will present preliminary results for a tool developed by Roth’s Cognitive
Computation Group, the Event Annotation Tool (EAT+). EAT+ uses advanced NLP technologies in conjunction
with large amounts of training data to annotate key textual passages (trigger words, actors, dates, places, etc.). The
technology underlying EAT+ will be outlined and success rates for detecting references to a representative set of key
civil unrest events (generic political attacks, kidnappings, assassinations, assembly of coercive forces, imposing a
curfew, declaring a state of emergency, restricting movement of citizens, etc.) will be reported. The final section of
the paper will report on SPEED’s efforts to harness the analytic power of SSP data through the use of various
composite measures of event intensity and event origins. This section will outline post WWII trends in civil unrest
and state repression.
Sean Newman (Fire Department of New York): How Adherence to Journalistic Standards
Positioned FDNY Watchline at the Forefront of Fire Service Analytical Intelligence Captain Newman’s presentation will show why the FDNY*s weekly intelligence newsletter, Watchline,
continues to add readers and garner praise from across the globe by following some basic rules of
journalism, such as concise headlines and lead sentences, brevity, story selection, and the collective
editing process. Other topics to be discussed are feature stories, deadlines and the importance of feedback.
Also, attendees will see how Watchline compares to some of Sherman Kent’ principles.
20
Stefania Paladini (Coventry University): Bayesian analysis in intelligence and social
sciences: a tool for all trades?” Bayesian analysis is one of the most versatile methods in social
sciences, with its application spanning from probability to finance to psychology. Incidentally, it is also
one which relevance in intelligence is well known and documented, at least since Sherman Kent’s times.
In a now quite famous text, Psychology of Intelligence Analysis, Heuer also provided some interesting
examples of its potential and its worthiness for the job. Also, he argued with reason, Bayesian reasoning
can significantly reduce the effects of cognitive biases, one of the main problems in intelligence. Still,
Bayesian analysis’ application, in intelligence as much as in social sciences, can be somehow problematic
or difficult, for a series of reasons – first of all, challenges linked to its technical complexity. It has been
proved that even people mathematically trained are generally poor in estimating conditional probabilities,
which are the objects of Bayes’ Theorem. Furthermore, some of its difficulties are linked to the quality of
the prior information available, as well as to the collection of suitable posterior ones. However, by
reviewing the available literature and by conducting some experiments in this sense it is the author's
impression that the instrument fails more than often due to the fact that it is used to address the wrong
questions - and not because of its internal weaknesses. The aim of this paper is to explore the issues
related to Bayesian analysis, to define when its use is appropriate and in which term, both in intelligence
and in social sciences and to provide suitable evidence.
Michael Robinson (Deloitte and Touche, LLP): Intelligence Analysis and Digital Forensics There are a large number of similarities between the fields of intelligence analysis and digital forensics.
Both disciplines collect data from a variety of sources, analyze data to form actionable intelligence, and
report information to stakeholders in an unbiased and timely manner. In the field of digital forensics,
analysis results in the formation of a report that can withstand legal scrutiny and be incorporated in legal
proceedings/trials. While there are similarities in the goals and objectives of the two disciplines, there are
resemblances in the current obstacles as well. Many of the challenges encountered in digital forensics are
also occurring in intelligence analysis, for example, there is a need to collect and process large amounts of
data (often terabytes) in a short amounts of time, there are a limited number of seasoned practitioners in
the field, there is a finite pool of resources, there is a need to provide qualitative indicators of reliability
and confidence to analyzed data, there is a need to provide information in an easily digestible format for
stakeholders that have limited time to read them, and there is a need to overcome misinformation that
appears from anti-forensics techniques. This paper/presentation will examine some of the subspecialties
within digital forensics, such as media analysis, mobile device forensics, and intrusion analysis, to
identify current obstacles and current solutions to these mounting problems. Comparisons and contrasts
between the fields of intelligence analysis and digital forensics will be made with an emphasis on
introducing solutions to overcome the current hurdles.
Patricia Rooney (National Ground Intelligence Center, US Army) and Peter Ronayne
(Federal Executive Institute): Neuroscience in Intelligence Analysis: Understanding How
the Brain Works and Its Impact on Analytic Decision Making: Intelligence analysis is neither
straightforward nor foolproof; it is both an art and a science. Whether pursuing strategic warning, current
intelligence or estimative intelligence, all analysts depend upon neuro-psychological capabilities that are
in constant tension and competition with one another. Most analysts know little about this neuro-tension
or the neurological processes that leave analysts vulnerable to deception in their daily analyses. This
session provides participants with (1) an understanding of the neurological tensions and cognitive
plasticity that analysts can draw upon for effective decision making; (2) key cognitive fitness tools to
keep the decision-making apparatus in tip-top shape; and (3) an understanding of intuition (art) and
rational, methodical (science) reasoning for analytic conclusions.
21
Gabriel Sebe (Bucharest University): Political Marketing Intelligence The aim of the present paper is to study the interaction in conceptual and applied terms between the
intelligence realm and the political marketing field; it is organized in four sections. The first section takes
into account two dimensions. The first focus on the succession series interaction between intelligence and
other fields conceptually related with the political marketing realm: journalism (Park, Lippmann, etc.),
public relations (Ivy Lee, Bernays, etc.), political science (Shotwell, Lasswell, etc.), international relations
(Shotwell, Mitrany, etc.), etc. The shift induced by Vannevar Bush’s paradigm and Shannon’s
information theory allows an integration, imposing a conceptual platform related with the informational
society concept; this is the second dimension. Now an explicit relationship between Levitt’s marketing
and Sheth political marketing mapping, Hunt’s resource/advantage theory and some main issues from the
intelligence field follows. Putting Kent’s contribution in this framework and taking into account the
Kendall/Kent debate, someone infers new perspectives on the knowledge transfer problem in terms of
interdisciplinarity, multidisciplinarity and transdisciplinarity. The second section introduces a
segmentation compatible with Downs’ political information and Machlup’s knowledge production,
adequated in terms of the intelligence cycle idea Boyd’s OODA framework. In this manner it becomes
possible to take into account two process-like dimensions. First the consideration of deliberation/decision
processes related with BI, CI, SI and their corresponding counterparts from the political marketing field.
Secondly, the differences and similarities concerning the meaning of ‚analysis’ as usable knowledge
transfer and fusion knowledge dedicated to support a competitive decision process. Beyond
terminological labels, the source classification problem, the relevance criteria, the measurement of
meaning paradigms, the transparency/opacity policy, the secrecy patterns, etc. and their associated
processes in terms of head building through education, instruction, trening form the basis of the analogies
necessary for conceptual transfer between these realms. In fact many problems treated in intelligence
terms can’t be conceived without a political marketing representation; the international relations
trade/conflict problem (Crucé, 1623; Polachek, 1980; etc.) is one example. Both intelligence and political
marketing fields are transdisciplinary; they cannot replace each other however together in a controllable
and rational knowledge fusion setting allow better solutions. The third section maps the analogies
between the intelligence field components and political marketing structured processes. Putting in
evidence some micro/macro principles it becomes possible to relate the specific knowledge
representations of the intelligence market conceived as the environment for both an intelligence economy
and a competitive political marketplace. Taking into account some preceding considerations, a certain
segmentation of research (fundamental, applied, practical, commercial) is put in correspondence with
distinct manners of learning generating different intellectual profiles in terms of theoretical/practical
capabilities. In this manner we enlarge the meaning for both usable knowledge and knowledge transfer
between theory and practice not only in conceptual, but also in human resource terms, giving a meaning
for the concept of intelligence capital. The fourth section concludes on the relevance of the ‚political
marketing intelligence’ paradigm from Boyd’s destruction and creation perspective.
22
Musa Tuzuner (Intelligence Studies Research Center; Turkish National Police Academy):
Multi-Level Cognitive Biases in the Chain of Intelligence Flow—From Source to
Consumers: Improving intelligence analysis has been attracting more attention in the last decade
because of the significant intelligence failure occurring in the world. Very few scholars have examined
the cognitive bias factors’ negative affects on the intelligence analysis. They believe that understanding
cognitive biases affecting intelligence biases would better serve to have efficient intelligence product for
the consumers. However, in order to produce better intelligence product, we need to understand the
Cognitive biases, not only in the intelligence analysis, but also in the chain of Intelligence flow. From our
intelligence field experience, we have found a flaw in looking for cognitive bias in the intelligence
analysis (Evaluators). By the time intelligence information reaches the analysis level, a cognitive bias is
already embedded when going from Informers to Collectors and so on. This is why detecting the
cognitive biases in the Evaluators stage is not adequate to produce efficient intelligence for the
consumers. In order to find where the biases originate, we have created a “Manual for Detecting
Intelligence Failure”. In order to detect multi-level cognitive biases in the chain of intelligence flow
Aydinli and I are collecting data (through interviews) from intelligence practitioners in Pakistan, Turkey,
Iraq, and Lebanon. After analyzing this dataset, we will classify the cognitive origins of the defects in the
intelligence flow: as individual origins of defects and institutional origins of the defects. Then we will
discuss how these collection, evaluation, and sharing defects can lead to intelligence failure.
Kathleen Vogel (Cornell University): Current Intelligence Reporting and the H5N1 Bird
Flu Virus: Insights from the Field of Science and Technology Studies In late 2011, virologists Ron Fouchier and Yoshihiro Kawaoka encountered a swarm of government and
public controversy from their creation of novel variants of the H5N1 bird flu virus. Prior to publication of
these experimental findings, the U.S. National Science Advisory Board for Biosecurity (NSABB)
reviewed Fouchier’s and Kawaoka’s scientific manuscripts. The NSABB unanimously recommended that
the, “conclusions of the manuscripts be published but without experimental details and mutation data that
would enable replication of the experiments.” The NSABB explained its justification as being based on
security concerns: “publishing these experiments in detail would provide information to some person,
organization, or government that would help them to develop similar viruses for harmful purposes.”
However, once the NSABB made its decision public, other flu virologists emphasized the difficulty of the
experiments and how these experimental results could lead to beneficial new medical treatments. Other
scientists, however, have sided with the NSABB and warned of the security dangers in publishing these
bird flu results. In the middle of this context, U.S. intelligence analysts have been tasked by their
superiors to provide up-to-date security assessments of these bird flu experiments as new information
about the experiments continues to emerge. These intelligence analysts have struggled to make sense of
the positions held by the NSABB and different scientific experts. These analysts have also encountered
challenges with how to gather and evaluate information in a timely fashion to better assess the role of
explicit and tacit (know-how) knowledge in these experiments. These intelligence assessments are needed
to better inform policymakers about whether to censor future logical research findings and how to think
about emerging biological science threats. This paper will discuss what happens when academic scholars
from the social science field of science and technology studies (S&TS) are brought into unclassified
dialogues with U.S. intelligence analysts as these analysts are writing their current intelligence reports on
these controversial bird flu experiments. This paper will illustrate how S&TS research can provide new
insights for assessing the security threat from these experiments and related biosecurity/WMD concerns,
how S&TS research can more broadly inform the analytic practices that underpin current intelligence
reporting, as well as challenges that exist in bringing S&TS ideas to bear on intelligence analysis.