Air Force Institute of Technology AFIT Scholar eses and Dissertations Student Graduate Works 3-23-2018 Improving Decision Support through Storytelling Tarah E. Coon Follow this and additional works at: hps://scholar.afit.edu/etd Part of the Leadership Studies Commons is esis is brought to you for free and open access by the Student Graduate Works at AFIT Scholar. It has been accepted for inclusion in eses and Dissertations by an authorized administrator of AFIT Scholar. For more information, please contact richard.mansfield@afit.edu. Recommended Citation Coon, Tarah E., "Improving Decision Support through Storytelling" (2018). eses and Dissertations. 1883. hps://scholar.afit.edu/etd/1883
74
Embed
Improving Decision Support through Storytelling · 2020. 3. 5. · improving decision support through storytelling. thesis . tarah e. cotton, captain, usaf . afit-env-ms-18-m-190
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Air Force Institute of TechnologyAFIT Scholar
Theses and Dissertations Student Graduate Works
3-23-2018
Improving Decision Support through StorytellingTarah E. Cotton
Follow this and additional works at: https://scholar.afit.edu/etd
Part of the Leadership Studies Commons
This Thesis is brought to you for free and open access by the Student Graduate Works at AFIT Scholar. It has been accepted for inclusion in Theses andDissertations by an authorized administrator of AFIT Scholar. For more information, please contact [email protected].
Recommended CitationCotton, Tarah E., "Improving Decision Support through Storytelling" (2018). Theses and Dissertations. 1883.https://scholar.afit.edu/etd/1883
DISTRIBUTION STATEMENT A. APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED.
The views expressed in this thesis are those of the author and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the United States Government. This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States.
AFIT-ENV-MS-18-M-190
IMPROVING DECISION SUPPORT THROUGH STORYTELLING
THESIS
Presented to the Faculty
Department of Systems and Engineering Management
Graduate School of Engineering and Management
Air Force Institute of Technology
Air University
Air Education and Training Command
In Partial Fulfillment of the Requirements for the
Degree of Master of Science in Cost Analysis
Tarah E. Cotton, BS
Captain, USAF
March 2018
DISTRIBUTION STATEMENT A. APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED.
AFIT-ENV-MS-18-M-190
IMPROVING DECISION SUPPORT THROUGH STORYTELLING
Tarah E. Cotton, BS
Captain, USAF
Committee Membership:
R. David Fass, PhD Chair
John J. Elshaw, PhD Member
Lieutenant Colonel Brandon M. Lucas, PhD, USAF Member
iv
AFIT-ENV-MS-18-M-190 Abstract
The role of decision support has emerged in the Air Force, Department of
Defense, and in the civilian sector. With information being more accessible than ever
before, the ability to analyze, interpret, and communicate information effectively and
efficiently has become essential. While extensive resources are expended on growing the
strength of decision support and analytical capability, the skills involved in presenting the
data to decision makers is under-developed.
Now more than ever, decision makers are being tasked with making huge
decisions in short periods of time. With a shrinking workforce and greater reliance on
automated systems, decision makers must use their limited time to rely on their trusted
advisors to interpret and communicate all of this information accurately and objectively.
While the analyst may understand the data, ultimately it is up to the decision maker to
make an informed decision. If we incorporate storytelling into decision support
presentations, we may improve the ability to request, defend, and justify resources within
the Air Force and DoD.
v
Acknowledgments
The research conducted in this paper could not have been possible without the
advisement, support, and patience of the primary research advisor, Dr. R. David Fass.
As in all accomplishments, it truly takes a village so I would like to thank my mentors,
Col Darnell Salley, and CMSgt Kendall Briscoe for seeing the potential in me that others
did not and who initially taught me that stories remind us of not only who we serve, but
why. Alas, nothing in my life would happen without the love and support of my wife,
Alisha and my dad and stepmother who always cheer the loudest even if I find myself in
last place. Finally, I would be remised not to mention the sacrifice of the individuals who
walked the path ahead of me, creating opportunities like attending AFIT available to
people like me. Thank you.
Tarah E. Cotton
vi
Table of Contents
Page
Abstract .............................................................................................................................. iv
Table of Contents ............................................................................................................... vi
List of Figures .................................................................................................................. viii
List of Tables ..................................................................................................................... ix
I. Introduction .....................................................................................................................1
General Issue ................................................................................................................1
Research Objectives .....................................................................................................2
** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). 1 indicates variable was measured in chart 1 (table) 2 indicates variable was measured in chart 2 (bar chart) 3 indicates variable was measured in chart 3 (narrative chart) Numbers in parentheses are Cronbach alpha coefficients
Table 4: Storytelling Correlations and Reliabilities
Variable 1 2 3 4 5 6
1. Focused Attention1 (4Qs) (0.91)
2. Believability1 (4Qs) 0.20 (0.92)
3. Willingness to rely on1 (3Qs) 0.41** 0.51** (0.91)
** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). 1 indicates variable was measured in the storytelling presentation 2 indicates variable was measured in the fact-based presentation Numbers in parentheses are Cronbach alpha coefficients
34
Hypothesis Tests
Hypothesis Test #1:
H1A: Visualizations with narrative elements are more aesthetically
pleasing than charts without
H1B: Visualization charts with narrative elements have higher perceived
usability than charts without
To test Hypothesis 1, the researchers utilized a repeated measures or ‘paired
samples’ t-test because the design was within participants, meaning all participants
contributed data for all of the conditions (i.e. all participants were shown all three
versions of the visualization charts). The pairs tested and results are listed in Table 5.
Based on the results of the paired samples test, there was no significant difference
in believability among the three sets of charts; however, there were differences in
perceived usability and aesthetics. Specifically, perceived usability in the table chart
(visualization 1) was lower than in the bar chart (visualization 2). Surprisingly, adding
narrative indicators to the annotated bar chart (visualization 3) did not improve perceived
usability. Both the bar chart (visualization 2) and the annotated line chart (visualization
3) generated significant results when compared to the table chart (visualization 1);
however, there was no difference in means of the bar chart (visualization 2) and the
annotated line chart (visualization 3). As a result, we reject both Hypothesis 1A and
Hypothesis 1B because narrative elements were only added to the third chart.
35
Table 5: Visualization Paired Samples Test
* Indicates significance at the .05 level
Hypothesis Test #2:
H2: Information presented in a storytelling context will receive more
focused attention from the audience than a fact-based presentation
Hypothesis test 2 was conducted in the same manner as Hypothesis test 1 using a
one-tailed t-test due to the directional hypothesis. Based on the significance indicated in
Table 6, the briefing presented in a storytelling context (Attention1) proved to garner
more focused attention than the fact-based presentation.
Hypothesis Test #3:
H3A: Information presented in a storytelling context is less believable
Beliveability1 Pearson Correlation -.134 .021 .101 Sig. (2-tailed) .273 .864 .407 N 69 69 69
Usability1 Pearson Correlation -.112 .057 .053 Sig. (2-tailed) .358 .643 .666 N 69 69 69
Aesthetics1 Pearson Correlation -.292* .049 .219 Sig. (2-tailed) .015 .689 .071 N 69 69 69
Beliveability2 Pearson Correlation .118 .050 -.146 Sig. (2-tailed) .333 .684 .232 N 69 69 69
Usability2 Pearson Correlation .386** -.170 -.203 Sig. (2-tailed) .001 .163 .095 N 69 69 69
Aesthetics2 Pearson Correlation .230 -.059 -.156 Sig. (2-tailed) .057 .632 .201 N 69 69 69
Beliveability3 Pearson Correlation .283* .054 -.296* Sig. (2-tailed) .018 .660 .014 N 69 69 69
Usability3 Pearson Correlation .479** .216 -.602** Sig. (2-tailed) .000 .074 .000 N 69 69 69
Aesthetics3 Pearson Correlation .449** .244* -.598** Sig. (2-tailed) .000 .044 .000 N 69 69 69
**. Correlation is significant at the 0.01 level (2-tailed).
*. Correlation is significant at the 0.05 level (2-tailed).
39
V. Conclusions and Recommendations
Chapter Overview
This chapter will answer the study’s research question as well as identify key
takeaways from the research effort. Specific findings as well as the research team’s
interpretations and conclusions are also included in the chapter.
Research Findings
Research Question: Does information presented in a narrative context improve
presentations as decision support tools?
Visualization Findings
Although the researchers were unable to determine that adding narrative elements
to visual charts positively affected the perceived aesthetics, usability, or believability of a
visualization, the findings did support the notion that tables are the least efficient
visualizations. Both the bar chart (visualization 2) and annotated line chart (visualization
3) outperformed the table chart (visualization 1) in every measured category. Even
though the difference in means of the bar chart (visualization 2) and the annotated line
chart (visualization 3) lacked significance, when asked to rank all three charts, 56.5% of
participants selected the annotated bar chart (visualization 3) as their number one
preference. The researchers were able to make some inferences from participant’s
feedback. Specifically, the participants commented that the annotated chart (visualization
3) “visually told the story” “highlighted the personal mission impacts” and was the
“easiest and quickly comprehend”.
Storytelling Findings
40
The researchers were also able to determine storytelling positively affected the
effectiveness of a decision support brief by increasing the audience’s focused attention.
We were also able to defend the use of storytelling against the assumption that
storytelling detracts from the credibility of a presentation. When asked to rate the videos
independently of each other there was no significant difference in the recorded means of
believability and willingness to rely. Additionally, when asked to compare the
memorability of the fact-based briefing to the storytelling briefing, on average,
participants scored the fact-based briefing a 2 or “fair”. Interestingly, even in negative
feedback about the storytelling briefing, the participants were able to recall specific
details indicating that whether they preferred the presentation or not, they did remember
it.
Conclusions of Research
Officers within the Secretary of the Air Force’s office of Financial Management
and Budget, the Operations branch (SAF/FMBO) widely known as the Engine Room, are
storytellers for the Air Force financial management community. A large part of these
officers’ responsibility is interpreting the results of Major Command (MAJCOM)
analysts, combining budgetary information from every reporting unit falling under their
purview, and creating a story explaining the Air Force’s situation. Much more than
“Power Point Rangers” quibbling over the appropriate shade of blue, the briefing support
they provide becomes strategic Air Force communications used to support and defend
budgetary requests. SAF/FMBO is not the only office in which airmen simultaneously
fill the role as of analyst, scripter, editor, and--at the lowest level of the Air Force
41
corporate structure--presenters. Due to time constraints, it forces them to create what
Knaflic (2015) refers to as a “Slideuments” or single documents that attempt to combine
the requirements of live presentation and written communication. These documents are
dangerous because the author loses control of the intended message and they fail both at
being a clear written report and at being an effective presentation. Often the result is
what is known as “Death by PowerPoint.”
While previous studies have examined the effectiveness of using business
analytics to drive decision-making (Brynjolfsson, Hitt, & Kim, 2011 & Moore, 2017)
none have simultaneously examined the effect of storytelling in military communication.
Comments and feedback from the presentation indicated that adding a storytelling
element touched on the human element of the fictional situation rather than numbers and
requirements. Invoking the human element “put a face on the need” and arguably created
an additional sense of duty to act. The criticism of the storytelling presentation called
into question credibility and ethics of using a story to persuade. One participant
commented that the presentation seemed “pandering” and others felt like the fictitious
SSgt was being used. The researchers believe those suspicions could be partially due to
the manner in which the presentations were shown. A prerecorded narrated PowerPoint
loses the human contact and specific nuances necessary to express the level of sincerity
that only face-to-face interactions can evoke. Additionally, each visualization included
the same bulleted text ensuring the same information could be clearly understood. The
overwhelmingly positive feedback for the annotated line chart’s ability to “easily
compare and contrast” (visualization 3) hints that the bulleted text was not needed.
Garnering additional support for the use of narrative attributes in visualizations.
42
In essence, decision support briefings are a call to action, whether it be for
resourcing requirements or implementing policy. If storytelling can enhance the decision
maker’s focused attention, then the presentation is more powerful and ultimately more
effective.
Recommendations for Future Research
The researchers believe limitations due to the manner in which the data was
collected plays a large role in the outcome of the study. Particularly, the use of a
prerecorded video limits the audience’s involvement in the presentation. Future research
could be conducted to determine how much physical presence impacts focused attention,
and whether the speaker is able to make stronger connections with the audience possibly
affecting the presentation’s believability, and the audience’s willingness to rely on the
presentation. This research would be extremely relevant to the Air Force given so much
crucial training is completed via computer-based training. Additional research could also
support recommended changes in the way formal school houses train financial
management officers to speak and present information in decision support contexts.
Even if every financial manager cannot become a world-class storyteller, understanding
the elements that create a story is easily translated to drafting and presenting information.
43
Appendix A. IRB Survey Approval
44
Appendix B. Online Survey Instrument
Attachment 1: Informed Consent
45
Attachment 2: Storytelling Section
46
47
Attachment 3: Data Visualization & Demographic Sections
48
49
50
51
Appendix C. Narrative Scripts
Attachment 1: Video 1 (Storytelling) Script
52
53
Attachment 2: Video 2 (Fact-based) Script
54
55
Appendix D. Storytelling PowerPoint Slides
Attachment 1: Video 1 (Storytelling) PowerPoint Slides
Slide 1
Slide 2
56
Slide 3
Slide 4
57
Slide 5
58
Attachment 2: Video 2 (Fact-based) PowerPoint Slides
Slide 1
Slide 2
59
Slide 3
Slide 4
60
Slide 5
Slide 6
61
Bibliography
Bumiller, E. (2010). We have met the enemy and he is PowerPoint. New York Times, 26, 1-1.
Campbell, J. (1949). The Hero with a Thousand Faces. NewWorld Library. Duarte, N. (2010). Resonate: present visual stories that transform audiences. Hoboken,
NJ: Wiley. Few, S. (2004). Data Presentation: TAPPING THE POWER OF VISUAL
PERCEPTION. Intelligent Enterprise, 13(7), 33. Fryer, B. (2003). Storytelling That Moves People. Harvard Business Review, 81(6), 51-
55. Gershon, N., & Page, W. (2001). What Storytelling Can Do for Information
Visualization. Communications Of The ACM, 44(8), 31-37. doi:10.1145/381641.381653
George, D., & Mallery, P. SPSS for Windows Step by Step: A Simple Guide and
Reference. 11.0 Update (4th ed.). Boston: Allyn & Bacon, 2003. Gliem, Joseph A. and Gliem, Rosemary R. “Calculating, Interpreting, and Reporting
Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales,” 2003 Midwest Research to Practice Conference in Adult, Continuing, and Community Education. Columbus OH, 2003
Knaflic, C. N. (2015). Storytelling with data: a data visualization guide for business
professionals. Hoboken, NJ: Wiley. Lee, B., Riche, N., Isenberg, P., & Carpendale, S. (2015). More Than Telling a Story:
Transforming Data into Visually Shared Stories. IEEE Computer Graphics And Applications, 35(5), 84-90.
McKee, R. (1997). Story substance, structure, style, and the principles of screenwriting.
London: Methuen. Obrien, H. L., & Toms, E. G. (2009). The development and evaluation of a survey to
measure user engagement. Journal of the American Society for Information Science and Technology, 61(1), 50-69
Segel, E., & Heer, J. (2010). Narrative Visualization: Telling Stories with Data. IEEE
Transactions on Visualization and Computer Graphics,16(6), 1139-1148. doi:10.1109/tvcg.2010.179
62
Seo, K., Lee, S., Chung, B. D., & Park, C. (2014). Users’ Emotional Valence, Arousal,
and Engagement Based on Perceived Usability and Aesthetics for Web Sites. International Journal of Human-Computer Interaction, 31(1), 72-87
Soh, H., Reid, L. N., & King, K. W. (2009). Measuring Trust In Advertising. Journal of
Advertising, 38(2), 83-104 “SPSS FAQ”. UCLA: Statistical Consulting Group. 14 Jan 2007. Retrieved on 08
February 2018. Tufte, E.R. (1983). The Visual Display of Quantitative Information. Graphics Press. Tufte, E. R. (2011). The cognitive style of PowerPoint: pitching out corrupts within.
Cheshire, Conn: Graphics Press. Wakeling, S., Clough, P., Wyper, J., & Balmain, A. (2015). Graph Literacy and Business
Intelligence: Investigating User Understanding of Dashboard Data Visualizations. Business Intelligence Journal, 20(4), 8-19.
Yau, N. (2013). Data points: visualization that means something. Hoboken, NJ: Wiley.
63
REPORT DOCUMENTATION PAGE
4. TITLE ANO SUBTITLE
Imp:oving Oecisic,n Support. t.h rough St.o r ~ -:Hing
&. AUTHOR(SJ Cottoa, Tarah E, Capt F = , Robert, D, PhD, USAF
7. PERFORM fNG ORGANIZATION NAM E(S) AHO AODRESS(ES)
Air For ce I ns t.ieuee of Tech.~ol ogy Graduace School of Engineering a nd Management. (AFIT/ EN) 2950 Hobson Way, Buildi ng 640 WPAE"B 02 45433- 886$
, . SPONSORlNG I MONITORlNG AGENCY NAM~S) ANO AOORESS(ES)
Form Approved 0MB No. 07044188
Sa. CONTRACT NUMBE.R
Sb. GRANT NUMBER
Se. PROGRAM ELEMENT NUM BER
Sd. PROJECT NUMBE.R
S&. TASK HUM.BER
St WORK UNIT HUM.BER
8. PE.RFORMING ORGANIZATION REPORT HUM.BER
AFTT-ENV-MS-18-M-190
Cose Analysis Direc t.o rat.e, Policy a nd Incegrat.ion Branch Col one ! M,:,m:c,e Neal
Dist.ribut.i.on St.av:ment. A. Approved f or P-.ioli c Re l ease ; Dist.=ibuci on Unlimit.eci
13. $ UPPLEM.EH TARY NOTE$ This mace =ial is d ecl!.ar-:d a work o f t.h e U. S . Gov-:rn."!lo:n::. and is not:. subjec::. t:.o c,:,pyri.ght. proeect:.ion i n t.he Unit.eci Scat:.es . 1'. AB STRACT
In a mi! i~ary enviror.ment. whe r e ::.he demand t.o de l i ver a nalyt:.ically rigor ous , ac::.iona b!e kno•,il!.edge has ou:;.paced ::.he skills and cools t:.o e ffec t.i v e ly cc•!l'ttl'.uni c at.e i n f o rmat:.ion, t:.he occasi onaL ~deat:.h by Po~e =Poi nt.• seems i nevit:.able .
As dat.a col!!.ec::.i on and a nalysis has evo:l!.veci, cc-tr.munica::.ion and p r e s -:nt.at.i c,n skil!.ls ha v e s::.agnat:.-:d, negl ecci ng O?port.unit:.ie s t.o c =eat.e po~e =f ul p=esent.a::.ions by capi ::.alizi ng on ::.he f usion o f hl.llt'.an c,:,nnect:.ion and dat.a visud.izacion t:.o persuad-: and i nfluence . The ::.rue proble.m is noc Po-~e =Point., buc ehe st.anda rdizat.ion of presencaeion t.echnique s ehae ~veneually l ead t.o misuse . Training decision suppo=c pe rsonne l in ehe are of st.or yeelling wich ciaea, ehae is t.o t.=ans form daca ineo vis ualized narraeives , is esseneial t.o evolving a neiq~at.ed brie f i ng p racci c es .
1S. SUBJECT TE.RM$ Sco=yt.e H i.ng, Daca Visualizat:.ion, D:cisi on Support., Co:nnr.mi cat.ion
1G. SECURITY CLASSIFICATION Of : 17.UMITATION Of ABSTRACT
1-~-•-•-PO=.-,---~ l g--A-.-.-,-RA-c-,--~1~-,-.-,.-.-.-0-,----< tiU
18. NUMBER 1S.a. NAME OF RESPONSIBLE PERSON Of PAGES Dr. R D.nid Fass, AfITIDN