Simon Buckingham Shum Connected Intelligence Centre • University of Technology Sydney @sbuckshum • http://utscic.edu.au • http://Simon.BuckinghamShum.net Towards Contested Collective Intelligence or… A tour of the CI design space for Hypermedia Discourse University of Melbourne • SWARM Project, 12 th Sept. 2017
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Simon Buckingham ShumConnected Intelligence Centre • University of Technology Sydney@sbuckshum • http://utscic.edu.au • http://Simon.BuckinghamShum.net
Towards Contested Collective Intelligence
or… A tour of the CI design space for Hypermedia Discourse
UniversityofMelbourne•SWARMProject,12th Sept.2017
Contested Collective Intelligence...
In wicked problems, there is no master worldview, ontology or logic
So disagreement is a necessary process and vital ingredient
We can disagree well or badly
CI tools should scaffold and improve this proess(e.g. amplify awareness of how stakeholders are framing the problem, reading
the signals, seeing connections, and judging success)
2De Liddo, A., Sándor, Á. and Buckingham Shum, S. (2012). Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work, 21, (4-5), pp. 417-448. http://doi.org/10.1007/s10606-011-9155-x
Dilemmasand
(partial)
Solutions
Dilemma
If everyone just talks with no structure, it’s hard to
the modelling of discourse / the discourse of modelling
…reading and writing networks of documents, concepts, issues, ideas and arguments
Buckingham Shum, S. (2006). Sensemaking on the Pragmatic Web: A Hypermedia Discourse Perspective. In: 1st International Conference on the Pragmatic Web, 21-22 Sept 2006, Stuttgart, Germany. ePrint: http://oro.open.ac.uk/6442
Hypermedia§ Modelling discourse relations§ Expressing different perspectives on a conceptual space§ Supporting the incremental formalization of ideas § Rendering structural visualizations§ Connecting heterogeneous content
Key ingredients of a Hypermedia Discourse approach
Dilemma
If users are required to structure their contributions to a CI repository, the effort must
provide tangible benefit (not just potential benefits to future stakeholders)
Solution(in small synchronous settings)
A skilled mapper resolves the cost-benefit tradeoff, adding
immediate value to the sensemaking
Issue Mapping (or in a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
Buckingham Shum, S. (2003). The roots of computer supported argument visualization. In P. Kirschner, S. Buckingham Shum, & C. Carr (Eds.), Visualizing Argumentation (pp. 3–24). London: Springer. ePrint: http://bit.ly/VizArgRoots
http://compendiuminstitute.net
Issue Mapping (or in a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
https://www.youtube.com/watch?v=pxS5wUljfjE
Issue Mapping (or in a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
this simple set of moves — combined with hypertext,
and mapping fluency —goes a long way…
UKResearchExcellenceFramework(REF)2014ImpactCase
Compendium software (open source)visual hypermedia for managing the connections between ideas flexibly
Deep acknowledgements:
Jeff Conklin CogNexus Institute
Al Selvin & Maarten Sierhuis NYNEX Science & Technology —> Bell Atlantic —> Verizon—> NASA
http://compendiuminstitute.net
20
Structure management in Compendium
§ Associative linkingnodes in a shared context connected by graphical Map links
§ Categorical membership nodes in different contexts connected by common attributes via metadata Tags
§ Hypertextual Transclusionreuse of the same node in different views
§ Templates reuse of the same structure in different views
§ HTML, XML and RDF data exports for interoperability
§ Java and SQL interfaces to add services
Compendium Institute: international communityhttp://CompendiumInstitute.net (now archived)
Selvin, S. & Buckingham Shum, S. (2015). Constructing Knowledge Art: An Experiential Perspective on Crafting Participatory Representations. Morgan Claypool. http://doi.org/10.2200/S00593ED1V01Y201408HCI023
HypermediaDiscoursefluencyatahighlevel
27
Mapping with IBIS Issue-templates to harvest the firm’s collective
intelligence on Y2K contingencies
Selvin, A.M. and Buckingham Shum, S.J. (2002). Rapid Knowledge Construction: A Case Study in Corporate Contingency Planning Using Collaborative Hypermedia. Knowledge and Process Management, 9, (2), pp.119-128.
28
Modelling organisational processes in Compendium using a Template
29
Completing a Compendium template
30
Generating Custom Documents and Diagrams from Compendium Templates
Selvin, A.M. and Buckingham Shum, S.J. (2002). Rapid Knowledge Construction: A Case Study in Corporate Contingency Planning Using Collaborative Hypermedia. Knowledge and Process Management, 9, (2), pp.119-128.
31
Using Compendium for personnel recovery operations planning
Conversational Modelling: real time dialogue mapping combined with model driven templates (AI+IA)
DARPA Co-OPR Project (PI: Austin Tate, AIAI, U. Edinburgh)http://www.aiai.ed.ac.uk/project/co-opr
Issues on which the I-X planning engine provided candidate Options
35
Mapping with IBIS to build a NASA science team’s collective intelligence for planetary geological exploration
Clancey, William J.; Sierhuis, Maarten; Alena, Richard L.; Graham, Jeffrey S.; Tyree, Kim S.; Hirsh, Robert L.; Garry, W. Brent; Semple, Abigail; Buckingham Shum, Simon J.; Shadbolt, Nigel and Rupert, Shannon M. (2007). Automating CapCom Using Mobile Agents and Robotic Assistants. In: 1st Space Exploration Conference: Continuing the Voyage ofDiscovery, 30 Jan-1 Feb 2005 , Orlando, FL, US. http://eprints.aktors.org/375
NASA: Mars Habitat field trials in Utah desert
NASA remote science team tools
Scientist (Mars)
Scientist (Earth)
Scientist (Earth)
Scientist (Mars)
Scientist (Earth)
Software Agent Architecture
(Mars)
Compendium used as a collaboration medium at all intersections: humans+agents reading+writing IBIS maps
Geology dialogue map between Earth-based scientists and ‘Mars’
Copyright, 2004, RIACS/NASA Ames, Open University, Southampton UniversityNot to be used without permission
Compendium activity plans for surface exploration, constructed by scientists, interpreted by software agents
Compendium science data map, generated by software agents, for interpretation by Mars+Earth scientists
Meeting Replay tool: Earth scientists can browse a (simulated) Mars crew’splanning meeting using Compendium
this simple set of moves — combined with hypertext and mapping fluency —
goes a long way…
BUT…
Dilemma
While co-located mapping is fine for ‘micro-CI’, can we scale this to support asynch. ‘macro-CI’?
Solution
Web-based IBIS mapping
Numerous IBIS-based web apps
http://oystr.cohttp://debatemapper.net
http://evidence-hub.net
http://litemap.net
http://cci.mit.edu/klein/deliberatorium.html
Where our tools fit… Given a wealth of documents…
46
Where our tools fit… and tools to detect and render potentially significant patterns…
47
Where our tools fit… and tools to detect and render potentially significant patterns…
48
Where our tools fit: we need ways to express interpretations
49
50
interpretation
interpretation
interpretation
interpretation
Where our tools fit: we need ways to express interpretations
51
interpretation
interpretationinterpretation
interpretation
interpretation
(a hunch – no grounding
evidence yet)
interpretation
Where our tools fit: we need ways to express interpretations
…and optionally make meaningful connections
52
predictscauses
interpretation
interpretationinterpretation
interpretation
interpretation
(a hunch – no grounding
evidence yet)
interpretation
Is pre-requisite for
53
prevents
predictscauses
interpretation
interpretationinterpretation
interpretation
interpretation
(a hunch – no grounding
evidence yet)Is inconsistent with
interpretation
challenges
Is pre-requisite for
…and optionally make meaningful connections
Potentially moving towards stories that make sense of the evidence… i.e. plausible narratives / arguments
54
Question
Answer
Supporting Argument… Challenging
Argument…
challengessupports
responds to
Assumption
motivates
Potentially moving towards stories that make sense of the evidence… i.e. plausible narratives / arguments
55
Question
Answer
Supporting Argument… Challenging
Argument…
challengessupports
responds to
Hunch
motivates
56
Question
Answer
Supporting Argument… Challenging
Argument…
challengessupports
responds to
Data
motivates
Potentially moving towards stories that make sense of the evidence… i.e. plausible narratives / arguments
Cohere demo (2011): web annotations with discourse connections
Structured deliberation and debate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
59
60
Structured deliberation and debate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
— web annotation of document (Firefox extension)
User/community-defined visual language
62
63
Structured deliberation and debate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
Comparison of one’s own ideas to others
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-Centric Learning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff
Does the learner compare his/her own ideas to that of peers, and if so, in what ways?
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. (2011). Discourse-centric learning analytics. 1st Int. Conf. Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr). ACM: New York. Eprint: http://oro.open.ac.uk/25829
What epistemic contributions are learners making in the community?
65
Rebecca is playing the role of broker,
connecting different peers’ contributions in
meaningful ways We now have the basis for recommending that
you engage with people NOT like you…
Evidence
Many users can make reasonable contributions to IBIS web apps,
without trainingBUT…
Dilemma
Asynchronous online mapping is tougher to curate: no on-the-
spot sensemaking from a mapper
Solution
Familiar looking web interfaces that guide users on how to
contribute good IBIS
Evidence Hub: structured storytelling for students, practitioners and researchers
Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net
A wizard guides the user through the submission of a structured story:• What’s the Issue?• What claim are you
making/addressing?• What kind of evidence
supports/challenges this?• Link it to papers/data• Index it against the core
themes
Evidence Hub: Argument Maps
Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net
The wizard then generates a structured IBIS tree showing evidence-based claims (and disagreements)
Evidence Hub: professional developmenthttp://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning
Issue
PotentialSolution
SupportingEvidence
(practitionerstory)
Dilemma:
Unstructured deliberation platforms provide no scaleable assistance in making sense of
Victoria Uren, Simon Buckingham Shum, Michelle Bachler, Gary Li, (2006) Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, Vol.64, 5, (420-445).
88
ClaiMaker returns a Lineage tree (the roots of a concept)
Dilemma:
Deliberation schemas focus attention on cold rationality, at the expense of social warmth
Solution
Addition of social channels in an IBIS mapping web app can
restore a sense of connectedness
L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067
Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
Solution
Addition of social channels in an IBIS mapping web app can restore a sense of connectedness
BUT…
But the group using a Ning discussion forum still outperforms Social-IBIS and Plain-IBIS
L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067
DebateDashboard
sociallyaugmentedCoheremapping
Ningdiscussionforum Cohere
But the group using a Ning discussion forum still outperforms Social-IBIS and Plain-IBIS
L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067
Writing is endlessly expressive and hard to improve on as a
medium for collective reflection/argumentation
(also a social process)
Dilemma:
But we would still like the machine to do some work for us in making sense of the state of
the CI process or product
Solution
NLP could move us beyond simple forum metrics, and help make sense of
the quality of contribution
Academic Writing Analytics: feedback on analytical/argumentative or reflective writing
Infohttps://utscic.edu.au/tools/awa
101
Highlighted sentences are colour-coded according to their broad type
Sentences with Function Keys have more precise functions (e.g. Novelty)
Analysis of texts:• Metrics: lengths of words, sentences, paragraphs, and statistics of these• Syllables: metrics at the word level based on syllables• Named Entities: e.g. names of People, Places• Statistics: e.g. noun-verb ratio• Vocabulary: compound words, occurrences at sentence, paragraph and document level• Expressions: epistemic, self-critique and affective compound words• Spelling: feedback on spelling and basic grammar• Rhetorical moves: in analytical and reflective writing• Complexity: measures of the complexity of words, sentences and paragraphs
Sheffield, UK not as sunny as yesterday - still warm
Greetings from Hong Kong
Morning from Wiltshire, sunny here!
See you!
bye for now!
bye, and thank you
Bye all for now
Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations?
Not at the start and end of a webinar…
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
-60
-40
-20
0
20
40
60
80
9:28
9:32
9:36
9:
40
9:41
9:
46
9:50
9:
53
9:56
10
:00
10:0
5 10
:07
10:0
7 10
:09
10:1
310
:17
10:2
3 10
:27
10:3
1 10
:35
10:4
0 10
:45
10:5
2 10
:55
11:0
4 11
:08
11:1
1 11
:17
11:2
0 11
:24
11:2
6 11
:28
11:3
1 11
:32
11:3
5 11
:36
11:3
8 11
:39
11:4
1 11
:44
11:4
6 11
:48
11:5
2 11
:54
12:0
012
:03
12:0
412
:05
Average Exploratory …
Discourse analytics on webinar textchat
…but if we zoom in on a peak…
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
Discourse analytics on webinar textchat
-100-50
050
100
9:28
9:40
9:
50
10:0
0 10
:07
10:1
7 10
:31
10:4
5 11
:04
11:1
7 11
:26
11:3
2 11
:38
11:4
4 11
:52
12:0
3 Classified as “exploratory
talk”
(more substantive for learning)
“non-exploratory”
…language is used in a manner more akin to “Exploratory Talk” (Neil Mercer)
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664