Human Social and Behavioral Variability: Scientific Challenges and Opportunities Adam Russell Bill Regli Discover DSO Day June 15, 2017 Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
Human Social and Behavioral Variability:Scientific Challenges and Opportunities
Adam RussellBill Regli
Discover DSO Day
June 15, 2017
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
2
“Imagine how much harder physics would be if electrons could think.”
- Murray Gell-Mann
“Imagine how much harder physics would be if electrons had feelings.”
- Richard Feynman
Why DARPA?
https://www.pinterest.com/explore/gary-larson/
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
3
Why DSO?
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) 4
Why are social systems “hard”?
https://xkcd.com/793/
5
Humans may present additional (unique?) complexity
molecular cellular organ individual group/family community society
GROUP
Not SelectedSelected
Mea
n N
PYD
ELT
(Rec
over
y - B
asel
ine)
-2
-4
-6
-8
-10
-12
-14
-16
-18
Mrazek, Alissa J. et al. Culture and Brain 1.2-4 (2013): 100–117http://www.businessinsider.com/shaq-simone-biles-picture-super-bowl-2017-2
Van Dongen et al. Sleep (2004) May 1; 27(3):423-33https://sportsscientists.com/2011/08/talent-training-and-performance-the-secrets-of-success/
http://www.cam-can.org/highlights/index.htmlCourtesy of Dr. Andy Morgan, Univ New Haven
Shariff et al. Psych Sci (2007), 18(9); 803-809http://www.technologyreview.com/view/538866/the-social-network-illusion-that-tricks-your-mind/
Mao et al. Nature (2017) Jan 13;8:13800
Henrich et al. Behav Brain Sci (2010) Jun;33(2-3):61-83
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
6
Humans may present additional (unique?) complexity
GROUP
Not SelectedSelected
Mea
n N
PYD
ELT
(Rec
over
y - B
asel
ine)
-2
-4
-6
-8
-10
-12
-14
-16
-18
Mrazek, Alissa J. et al. Culture and Brain 1.2-4 (2013): 100–117http://www.businessinsider.com/shaq-simone-biles-picture-super-bowl-2017-2
Van Dongen et al. Sleep (2004) May 1; 27(3):423-33 https://sportsscientists.com/2011/08/talent-training-and-performance-the-secrets-of-success/
http://www.cam-can.org/highlights/index.htmlCourtesy of Dr. Andy Morgan, Univ New Haven
Shariff et al. Psych Sci (2007), 18(9); 803-809http://www.technologyreview.com/view/538866/the-social-network-illusion-that-tricks-your-mind/
Mao et al. Nature (2017) Jan 13;8:13800
Henrich et al. Behav Brain Sci (2010) Jun;33(2-3):61-83
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
7
http://psychsciencenotes.blogspot.com/2010/09/assume-cow-is-sphere.html
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
8
What do we need?
http://boldarc.com/produkter/vr-ar-och-plattskarm/Adapted from Evolution, 2007 Cold Spring Harbor Laboratory Press
Courtesy of Josh Plotkin, Univ Penn - 2017http://www.ugrad.cs.ubc.ca/~cs414/notes/sampling.html
https://www.pinterest.com/pin/509962357776759635/https://en.wikipedia.org/wiki/Open_science
Gieannetto, D. and Heydari, B. (2015) Nature: Scientific Reports | 6:21870 | DOI: 10.1038/srep21870
Slide shots courtesy of Ed Wack, MIT-LL
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
9
What do we have?
PREDICT:• Post-hoc • Little generalizability
DESIGN:• Under-powered• Single method
COLLECT:• Small lab studies• Big correlational data
ANALYZE:• Inappropriate
statistics• Overfitting
REPRODUCE:• Misaligned incentives• Barriers to sharing
MODEL:• Narrow modeling• Narrow teaming
Many challenges are technical, many are cultural, some are both
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) 10
One result
http://www.winchestermysteryhouse.com/
11
Next Generation Social Sciences (NGS2)
Program Concept• Build a new capability (methods,
models, tools, and a community of researchers) for rigorous, reproducible experimental research at scales necessary to understand emergent properties of human social systems
Technical Objectives• Use the exemplar challenge of “collective
identity” (CI) to develop and demonstrate new approaches for predictive modeling, experimentation, and reproducibility in social science
• Develop and demonstrate the impact of these approaches for accelerating discovery and understanding of human social behavior
Model
Predict
DesignCollect
Analyze
Reproduce
http://www.fbicaaa-bham.org/page-1298796; Lada Adamic, personal webpage
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
12
The road to a program: Early “Exploration”
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
13
The road to a Program: Request for Information
http://blog.executivebiz.com/2016/09/darpa-seeks-info-on-simulation-platform-development-to-validate-social-science-research-methods/
http://www.psychologicalscience.org/publications/observer/obsonline/darpa-seeks-information-on-experimental-falsifiability.html#.WRmdvfnyvRY
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
14
The road to a Program: Meet George Heilmeier
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
- What are you trying to do? Articulate your objectives using absolutely no jargon.
- How is it done today, and what are the limits of current practice?
- What is new in your approach and why do you think it will be successful?
- Who cares? If you succeed, what difference will it make?
- What are the risks?
- How much will it cost?
- How long will it take?
- What are the mid-term and final “exams” to check for success?
15
The road to a Program: The BAA
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) 16
More roads are needed…
http://www.darpa.mil/news-events/socialsim-proposers-day http://science.dodlive.mil/2014/09/28/math-the-key-to-compiling-science-data/http://www.darpa.mil/news-events/2017-04-06 http://www.darpa.mil/news-events/next-generation-social-science-proposers-day
http://www.darpa.mil/program/2016-11-28http://www.darpa.mil/news-events/2017-04-07
17
Challenges
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
18
Challenge 1: “Big” Ethnography
https://medium.com/ethnography-matters/why-big-data-needs-thick-data-b4b3e75e3d7
How do we get here?
“Humans are animals suspended in webs of meaning they themselves have spun.” - Clifford Geertz
Lada Adamic, personal webpage
Insert hand clasp and arm wrestle – which is
which?
http://basementrejects.com/review/west-side-story-1961/ Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
19
Challenge 2: AI Coaches
How can we use human variability to
design for/thrive in complexity?
What does a machine need to know about you to be a coach and not just a tool?
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
20
Challenge 3: Quantifying Human Entanglement in Complex Environments
Network of Facebook friendships, Emilio Ferrara 2015https://taraingalls.wordpress.com/2013/07/17/is-social-media-ruining-relationships/
https://en.wikipedia.org/wiki/Ecological_systems_theoryDistribution Statement “A” (Approved for Public Release, Distribution Unlimited)
21
Challenge 4: Designing Un-gameable Games
https://www.nap.edu/read/18622/chapter/5#81https://policytensor.com/2013/08/21/asymmetric-escalation-games-and-the-taiwan-dilemma/
From this:To this:
https://ourworld.unu.edu/en/systems-thinking-and-the-cobra-effect
“Neglect of the qualitative dimension often renders social-scientific research deeply problematic.”
– Crossley and Edwards, 2016
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited) 22
Moving towards “Solution-Oriented Social Sciences”
http://openeducationresearch.org/2009/01/pasteurs-and-edisons-quadrants/
23
From Topics to Goldilocks Problems
B. Clearly hard
Etsy, TheArtifulBookworm
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
A. Clearly important
C. Clearly measurable progress
24
Where are the Goldilocks Problems for the Noosphere?
How can we shape what happens?
What will happen next?
Micro-scale
What has happened?
What is happening?
Meso-scale
Macro-scale
Prediction/Correlation
Explanation/Causality
“If you do not know how to ask the right question, you discover nothing.”
- W. Edwards Deming
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)