DLI2 All Projects Meeting, Stratford Upon Avon 1 Paul Gorman, MD Lois Delcambre, PhD David Maier, PhD
Jan 09, 2016
DLI2 All Projects Meeting, Stratford Upon Avon 1
Paul Gorman, MD Lois Delcambre, PhDDavid Maier, PhD
DLI2 All Projects Meeting, Stratford Upon Avon 2
Bundles in the wild………..Observational team:
Paul GormanJoan AshMary LavelleJason Lyman
…………..Bundles in captivityComputer science team:
Lois DelcambreDave MaierShawn BowersLongxing DengMathew Weaver
DLI2 All Projects Meeting, Stratford Upon Avon 3
(Wild) Bundles
DLI2 All Projects Meeting, Stratford Upon Avon 4
(Wild) Bundles
DLI2 All Projects Meeting, Stratford Upon Avon 5
(Wild) Bundles
DLI2 All Projects Meeting, Stratford Upon Avon 6
(Wild) Bundles
DLI2 All Projects Meeting, Stratford Upon Avon 7
(Wild) Bundles
• manage information for diverse, complex tasks• contain selected, collected, structured, annotated• are often used in settings with:
– high uncertainty– low predictability– potentially grave outcomes– time & attention are highly constrained
DLI2 All Projects Meeting, Stratford Upon Avon 8
(Wild) Bundles
• There is benefit in creating (active processing of information)
• There is benefit in reusing (trigger memory)
• There is benefit in sharing (establish collective, situated awareness)
DLI2 All Projects Meeting, Stratford Upon Avon 9
Let’s take a trip to the ICU
DLI2 All Projects Meeting, Stratford Upon Avon 10
Given….
• bundles are everywhere! • access to bundles provides access to
important information• information in bundles is often copied from
other information sources
• we can keep copied/referenced information linked through the use of marks
DLI2 All Projects Meeting, Stratford Upon Avon 11
The superimposed and base layers with marks
Superimposed Layer
BaseLayer
Information Source1
Information Source2
Information Sourcen
…
marks
DLI2 All Projects Meeting, Stratford Upon Avon 12
(Captive) Bundles
• SLIMPad - a scratchpad application to create bundles but….with referenced information connected to the underlying source data
• helping us explore architectural issues for building superimposed applications
• motivating definition of a metamodel to represent information with mappings to transform
• inspired by the observational work (but not focused on a specific medical task)
DLI2 All Projects Meeting, Stratford Upon Avon 13
SLIMPad demo
DLI2 All Projects Meeting, Stratford Upon Avon 14
The SLIMPad Architecture
XMLViewer
userMS
PowerPoint
MSExcel
IEExplorer
XML Documents
PPT Files
Excel Spreadsheets
Web Pages
Superimposed Information Management
Mark Management
Generic Technology
PDFViewer
PDF files
SLIMPad
DLI2 All Projects Meeting, Stratford Upon Avon 15
The architecture for managing SLIMPad’s superimposed information.
Superimposed Information Management
SLIMPad
Java Objects(ActiveX)
SLIMPadAPI
(ActiveX)
TRIM Store(Java) RDF
Triples
creates andmaintains
DLI2 All Projects Meeting, Stratford Upon Avon 16
Generic Representation of Information
Topic Map
Topic Map Defintions
Topic Map Instances
XML
DTD
XML Document
Metamodel
SuperimposedLayer
Basic Set of Abstractions
Model Constructs and Relationships
Schema-LevelData
Instance-LevelData
DLI2 All Projects Meeting, Stratford Upon Avon 17
Topic Map Example
Painting Painterby painter
Influenced by
“Captive” “Paul Klee”by painter influenced by
“Francisco de Goya”
“1914”by painter
mentioned biographyreferenced
referencedhttp://...
biography biography
http://...http://...
critiqued
critiqued
mentioned
http://...
http://...
DLI2 All Projects Meeting, Stratford Upon Avon 18
Portion of Structured-Map Model as RDF
(instanceOf, “TopicType”, Construct)(instanceOf, “TopicInstance”, Construct)
(instanceOf, “ttypename”, Connector)(domain, “ttypename”, TopicType)(range, “ttypename”, String)(domainMult, “ttypename”, “*”)(rangeMult, “ttypename”, “1”)
(instanceOf, “topic_instOf”, Conformance)(domain, “topic_instOf”, TopicInstance)(range, “topic_instOf”, TopicType)(domainMult, “topic_instOf”, “*”)(rangeMult, “topic_instOf”, “1”)
TopicType
ttypename : String
TopicInstance
*
1
<<conformance>>
topic_instOf
DLI2 All Projects Meeting, Stratford Upon Avon 19
(instanceOf, “painting_tt”, TopicType)(ttypename, “painting_tt”, “painting”)(instanceOf, “painter_tt”, TopicType)(ttypename, “painter_tt”, “painter”)
(instanceOf, “byPainter_rt”, TopicRelType)(relType, “byPainter_rt”, “by painter”)(topicType1, “byPainter_rt”, painting_tt)(topicType2, “byPainter_rt”, painter_tt)
(instanceOf, “biography_at”, AnchorType)(anchorRole, “biography_at”, “biography”)(topicType, “biography_at”, painter_tt)
Representing SchemaTopic Types (schema):painting, painter
Topic Rel Types (schema):by painter
Anchor Types (schema):biography
painting painterby painter
biography
DLI2 All Projects Meeting, Stratford Upon Avon 20
Representing Instance(instanceOf, “painter1”, TopicInstance)(title, “painter1”, “Paul Klee”)(topicInsID, “painter1”, “5”)(topic_instOf, “painter1”, painter_tt)(instanceOf, “painting1”, TopicInstance)(title, “painting1”, “Captive”)(topicInsID, “painting1”, “19”)(topic_instOf, “painting1”, painting_tt)
(instanceOf, “byPainter1”, TopicRelInst)(rel_instOf, “byPainter1”, byPainter_rt)(topicIns1, “byPainter1”, painting1)(topicIns2, “byPainter1”, painter1)
(instanceOf, “biography1”, AnchorInst)(anchor_instOf, “biography1”, biography_at)(address, “biography1”, a1)
(instanceOf, “a1”, Address)(markID, “a1”, “URLMarkManager@954308545”)
Topic Instances (instance):Paul Klee, Captive
Topic Relationship (instance):a by painter relationship
Anchor (instance):a biography anchor
Address (instance):mark to url
DLI2 All Projects Meeting, Stratford Upon Avon 21
Three Basic Types of Mappings
Model1
Schema1
Instances1
Source Target
Mapped
Converted
Converted
Mapped
Converted
Mapped
Converted
Converted
Inter-Model
Inter-Schema
Model-to-Schema
Model2
Schema1
Instances1
Model1
Schema1
Instances1
Model1
Schema1
Instances1
Model1
Schema2
Instances1
Model2
Schema2
Instances
DLI2 All Projects Meeting, Stratford Upon Avon 22
(CS Team) Plans
“Schema” for bundles: through templates that predefine certain bundles and labels, and possibly predefine scraps. less restrictive than trad. schema.
Sharing: a bundle that belongs to multiple pads, a pad with multiple simultaneous users, and interchange of pads and bundles between users.
“Scoping” of base layers and marks over those layers. There are several general issues for modeling and management of superimposed information that have emerged from this work and other efforts.
DLI2 All Projects Meeting, Stratford Upon Avon 23
(CS Team) Plans (cont.) Distribution: more than a single mark manager. Is there a
single space of marks or must applications keep track of which manager handles a particular mark?
Mark Maintenance:when base layer changes New Kinds of Sources and Marks: beyond XML, PDF, MS
Word, MS Excel, MS PowerPoint and HTML. We are also interested in marks into a “base” layer that is actually superimposed information for another application.
Application Interface: try a more query-like API. Also, consider an API generator
• Superimposed Information Mapping/Conversion: applying and extending our metamodel.
DLI2 All Projects Meeting, Stratford Upon Avon 24
www.cse.ogi.edu/footprints
• demos - including the QTVR of the ICU (with toys) and SLIMPad
• personnel• project description• papers
– “Bundles in the Wild: Tools for Managing Information to Maintain Situation Awareness”
– “Bundles in Captivity: An Application of Superimposed Information”
– papers discussing superimposed information