a Model-driven development methodology for 3D User Interface for Information Systems in a Principle based way

Sectio

A Model-Driven Approach for Developing 3D User Interfaces

of Information Systems in a Principle–Based WayJuan Manuel González Calleros

PhD Public Defense

February 4, 2010 - LLN

February 4, 2010 - LLN PHD Public Defense

Virtual Reality

Augmented Reality

Windows Vista

Locate your sit in a stadium

Outline

1. Introduction2. State of the Art3. Methodology

1. Ontology2. The Method3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology

4. Validation5. Conclusion

February 4, 2010 - LLN PHD Public Defense 5

What is a 3D User Interfaces?

Augmented Reality Virtual Reality

3D User Interface

3D desktop environments

Why 3D User Interfaces?

Increase user satisfaction.

Improve cognitive perception

Indices sense of (tele)presence

3DUIs are not automatically superior or inferior to

2DUIs.

Why NOT 3D User Interfaces?

Decrease user performance

Increase manipulation complexity for an average user

Are not appropriate for any task

Are hard to evaluate for their usability

Focus of the dissertation

Information systems (data, process, resources)

Administrative tasks (routine, repetition)

Interaction styles (form filling, multi-windowing, direct manipulation, iconic interaction, graphic interaction, multimedia interaction, and 3DUIs)

Thesis Statement

In this thesis we argue that developing 3DUIs for Information Systems is an activity that would benefit from the application of a methodology which is typically composed of:

1. a set of models gathered in an ontology, 2. a method manipulating the involved models

based on guidelines, 3. a language that express models in the

method.Define a model-driven approach for structuring

the development life cycle of a Three-Dimensional User Interface of an Information System in a principle based way.

Comparative analysis

• 3DUI development methodologies

• Language• Transformation

Engines

• 3D Toolkits

• Models

• Language

• Methods

• Software tools

Requirements

Methodological (7)

Ontological (3)

Language (1)

Software tool support (1)

Methodological support for the development life-cycle of 3DUI for Information Systems

Outline

• 3D UI as an specialization of 2D

• Benefits:

• Relying on existing 2D UI by specializing

• Factoring out 3D vs 2D UIs

3D Rendering of a 2D GUI

VUITOOLKIT [Moli08]

• Shortcomings:• Adding another modality breaks the factoring

out• Hard to provide multiple representations of

3D objects• No full separation of concerns

• Shortcomings:• Adding another modality breaks the factoring

out• Hard to provide multiple representations of

3D objects• No full separation of concerns

Support for different

representations

Support for basic Haptic Interaction

Outline

Method Outline

Task and Domain Model

Model to Model

Abstract UI Model

Model to Model

Concrete UI Model

Code Generatio

Final UI

Control

Physical Control Software Control

Physical interaction object 2D 3D

Model to Model

Abstract UI Model

Model to Model

Concrete UI Model

Code Generatio

Final UI

• A structured catalog of transformation rules that form a body of design knowledge that can be reused in any 3D method

Code Generatio

Final UI

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

A set of Principles were added to the method

•Guidelines•Task patterns•Canonical list of task types

Canonical list of task types

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Facet Selection

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Select Element

Slider

AIO Selection

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Graphical representation selection

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Automatic guidelines evaluation

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Outline

1. Ontology2. Method3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology

Language Engineering Approach

Semantics – Meta Models, UML Class diagrams

SyntaxAbstract – XML SchemaConcrete – XML

Stylistics – Different graphical representations of the concepts

UsiXMLStructured accordingly to the Model Driven

paradigmUsiXML relies on a transformational

approach UsiXML allows the modification of the

developments stepsUsiXML allows reusing parts of previously

specified UsiXML is openFollows a Language Engineering Approach

Outline

1. Ontology2. Method 3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

IdealXML

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

TransformiXML

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

UsabilityAdvisor

Based onGuidelines

Model to model

Abstract UI Model

Based onGuidelines

Model to Model

Concrete UI Model

Usability Advisor

AutomaticEvaluation

Refined Concrete UI

Code Generatio

3D User Interface

Outline

1. Introduction2. State of the Art 3. Methodology

1. Ontology2. Method 3. Towards a UIDL for 3DUI 4. Software for Supporting the Methodology

Case Study

Complexity

Man/Month

LOC Context

Polling System

+ 6 6, 000VRML

Information Systems

Student TrainerSystem

+ 6 3, 000X3D

Information Systems

Haptic Browser

++ 4 (collaboration)

0 Information SystemsVisual Impair users

AHMI +++ 48 2, 000X3D10, 000 +Open GL

Information SystemsAeronautics

• Polling System. Devoted to the development of an opinion polling system.

• Hapget = 3D widget + haptic feedback

• Haptic Browser- Google web site. Rendering of web site in the haptic web browser.

• The AFMS is a text-only system that calculates the exact route of the aircraft along a given list of waypoints considering possible altitude, time or speed constraints.

• After a trajectory is generated and successfully negotiated with ATC the AFMS can generate input commands for the autopilot and thus control the aircraft in space and in time.

• The AFMS application is controlled via the Advanced Human Machine Interface (AHMI), the main application of the target system to be investigated in the HUMAN project

• Problems:• Discontinuity with

the rest of the environment

• Difficult to modify due to its Complexity

• Usability guidelines• Widgets have a 3D

nature in the context of a cockpit

• Our goal is to provide a 3D Advanced Human Machine Interface.• New set of CUI models• Consolidation of

existing models• New application

domain• …

External evaluation

SYSUSE INFOQUAL INTERQUAL OVERALL

Metric

CSUQ Perceived Methodology

External evaluation

• Final rendering – Feedback from the haptic browser• Difficult recognition of some hapgets.• Lack of user preference adaptation• Limited exploitation of the haptic characteristics• Long sentences to provide a value• Size of hapgets• Speech recognition problems• Sound become annoying• Slow loading

Outline

1. Introduction2. State of the Art 3. Methodology

1. Ontology2. Method 3. Towards a UIDL for 3DUI 4. Software for Supporting the Methodology

4. Validation 5. Conclusion

We introduced a 3DUI Development Methodology articulated on three axes:

Models

Method

Language

Models (More than 200 attributes, 90 classes, 100 relationships)

Task: Canonical list of task types and guidelines for task

modeling.

Domain Model: Improving existing Domain Model that will be

considered for the NexOF-RA the European standardization

process

AUI: Improving existing AUI Model that will be considered for

the NexOF-RA The European standardization process

CUI: Enriching CUI Model by adding haptic support,

separation of concerns 2D vs 3D by specialization. 3D support.

Hapgets (set of 3D widgets enhanced with haptic feedback)

DomainModel

DomainSubModel

+subModelId

DomainElement

DomainItem

+itemId+itemType

DomainCollection

+collectionId+collectionType

Constraint

+condition

RelevanceExistence

DomainRelationship

FunctionalDependency

+semanticFunction

ValueConstraint

ReplicationConstraintCardinalityConstraint

Domain Model

AUI Model

Integrates 3DUI in the context of MBUI development Incubator group

Integrates new concepts to their ontology based on UsiXML

http://www.w3.org/2005/Incubator/model-based-ui/wiki/Main_Page

http://forge.morfeo-project.org/wiki_en/index.php/Interactive_Application_Models

Methodological aspects adheres to the MDE paradigm. Models and transformations are explicitly defined and used (Around 85 transformations rules) Step 1: Task and concepts (T&C) definition

Task and Concepts Consolidation Selection of task types, items and user categories Guidelines of best-practices for structuring task models

Step 2: From T & C to Abstract UI (AUI) Model Identification of abstract UI structure Selection of AIC Spatio-Temporal arrangement of abstract interaction objects Definition of abstract dialog control Derivation of AUI to domain mappings

Step 3: From AUI to CUI Model Reification of AC into CC Selection of CICs Selection of the graphical representation of the CIC

Step 4: From CUI to FUI Model

All aspects are stored in UsiXML files that can be exchanged, shared, and communicated between stakeholders (designers, developers, and end users).

Advantages of this language were discussed Modifiability Complexity Rigorous. Reasoning. Processable.

Adopt a Language Engineering Approach Semantics – Meta Models Syntax

Abstract – XML Schema Concrete – XML

Stylistics – Different graphical representations of the concepts Our contribution is independent of UsiXML

Five Software modules are used to support the methodology. Conceptual definition of the tools. Supervising of their developmentTesting with our case studies the performance

of the toolsFinal rendering (48 men/month, more than

10, 000 LOC)

Benefits from our methodology

MDA Expected Benefits

Reducing the gap between requirements and implementation

Developer coordination. Dividing the system into several parts and defining interfaces between them

Well-structured systems. This facilitates implementation itself as well as maintenance.

Planning on adequate level of abstraction

Improved communication by visual models

Validation: (Semi-)Formal modeling languages enable automatic validation of the design

Documentation: Models can be used as documentation when maintaining the system.

Platform-independence. At least serve as starting point when implementing the system for a different platform

Enhanced productivity.

Expert knowledge can be put into the code generator

Reduction of errors

Knowledge about creation of modeling languages

Frameworks and tools

Maintenance of modeling language and transformations

Reuse of meta-models and transformations

Future WorkExternal evaluation should be targeted to three:

designers, developers and users:User testing with control experiments in labsDesigner and developers

Extend the set of 3DUIs components. Integrated development Environment Explore Other dimensions of the Cameleon

frameworkExplore other context of use

Learning ObjectsAugmented reality

Acknowledgements

2004 20092007 20082005 2006

•UsiXML Consortium

•The Information Systems Research Unit

•The ALBAN program

•The CONACYT program

•The SIMILAR project

•The Human project

•The NexOF project

•The Itea 2 UsiXML project

•The Promep project

_____________PhD_____________________DEA_______

a Model-driven development methodology for 3D User Interface for Information Systems in a Principle based way

domain model model

abstract ui model model

llnphd public defense

llnphd public defense

d method

modeldriven approach

d objects

d rendering

Education

Design Methodology of Camshaft Driven Charge Valves for ...

Metrics-Driven Design -...

Leverage SWIFT methodology for results-driven Oracle ... ›...

FIRST-PRINCIPLE DATA-DRIVEN MODELS FOR ASSESSMENT OF...

MOD-33 Principle, Criteria and Methodology: Pacific...

U-Slot Patch Antenna Principle and Design Methodology ...

CloudTeams Methodology: a Roadmap for Customer-Driven...

Risk-Driven Emergency Management A Principle of Emergency...

Peelian Principle in a Data-Driven World, by The R Simmons.....

Mockup-driven Fast-prototyping Methodology for Web...

Toward a Methodology for Manufacturability-Driven Design...

Recovery-Driven Design: A Power Minimization Methodology ...

MINISTRY MATTERS Principle Driven: No Foolin’ April 1,...

A Methodology for Data-Driven Decision-Making in Last Mile.....

A Data-Driven Methodology for Dynamic Pricing and Demand...

Principle driven scaling - How to grow value without growing...