Rome, Ing Ind 15 Workshop - Vr3lab Team Research Presentation - Speaker: M. Gattullo

Augmented Reality

in industrial environmentsWorkshop ING-IND/15

Roma, 23-24/01/14

VR3Lab

Politecnico di Bari

Vr3lab – Politecnico di Bari

Virtual Reality and Reality Reconstruction Lab

• Giuseppe Giuseppe Giuseppe Giuseppe MonnoMonnoMonnoMonno (Full professor, Director of DMMM)

• Antonio E. Uva Antonio E. Uva Antonio E. Uva Antonio E. Uva (Associate professor)

• Michele Michele Michele Michele FiorentinoFiorentinoFiorentinoFiorentino (Assistant professor)

• SaverioSaverioSaverioSaverio DebernardisDebernardisDebernardisDebernardis (PhD Student, currently at VRAC)

• Michele Michele Michele Michele GattulloGattulloGattulloGattullo (PhD Student)

• Michele Di Donato Michele Di Donato Michele Di Donato Michele Di Donato (Researcher)

• Ernesto Ernesto Ernesto Ernesto CarrabbaCarrabbaCarrabbaCarrabba (Researcher)

Our Vision

Augmented RealityAugmented RealityAugmented RealityAugmented RealityMotivationsMotivationsMotivationsMotivations

• Low cost devices

• Technical info where needed

• Up to date data

Main issues in industry

?1. Readability with HWDs

and projection

2. Tracking in industrial

environment

3. Ergonomic user interface

4. Fast and easy AR

authoring

1. Readability in AR

AuthoringInterfaceTrackingReadability

Overview

Current literature

• Contrast

• Background texturization

• Device technology (OST or VST HWDs, projection)

Our added value: industrial Our added value: industrial Our added value: industrial Our added value: industrial constraintsconstraintsconstraintsconstraints

• Color coding (e.g. ASME, ISO)

• Workplaces lighting (e.g. ISO, IESNA)

• No control on background

AuthoringInterfaceTrackingReadability

Study 1: Survey on text style with OST

DOEDOEDOEDOE

• Optical See-Through HWD

• Industrial backgrounds

• Illuminance 400 lx

Main resultMain resultMain resultMain result

• Billboard and outline are better than plain text

Fiorentino, M.; Debernardis, S.; Uva, A. E. & Monno, G.

Augmented reality text style readability with see-through head-mounted displays in industrial contextPresence: Teleoperators and Virtual Environments, The MIT Press,

2013, 22, 171-190

AuthoringInterfaceTrackingReadability

AR

Study 2: OST vs. VST

DOEDOEDOEDOE

• Not textured backgrounds

• Illuminance 400 lx

• Standard colors (ISO)

Main resultsMain resultsMain resultsMain results

• OST slightly better than VST

• Colors in the billboard with white text

• White text/blue billboard as default

Debernardis, S.; Fiorentino, M.; Gattullo, M.; Monno, G. & Uva, A.Text Readability in Head-Worn Displays: Color and Style Optimization in Video vs. Optical See-Through Devices.IEEE transactions on visualization and computer graphics, 2013

AuthoringInterfaceTrackingReadability

Study 3: Contrast limits

DOEDOEDOEDOE• 1000 lx vs. 4000 lx

• Variable outline width

Main resultsMain resultsMain resultsMain results

• VST better than OST

• OST not always usable (e.g. fine machining)

• Outline effective only for VST

• 1px outline optimal for VST

Gattullo, M.; Uva, A.; Fiorentino, M. & Monno, G.Effects of Lighting on AR Text Readability in Industrial Environment.IEEE transactions on visualization and computer graphics, submitted for publication

AuthoringInterfaceTrackingReadability

0

0,5

1

1,5

2

2,5

3

3,5

0 1 2 3 4 5 6 7 8 9 10

outlin

e px/

text

px

ratio

outlin

e px/

text

px

ratio

outlin

e px/

text

px

ratio

outlin

e px/

text

px

ratio

ooooutlineutlineutlineutline widthwidthwidthwidth

BILL.

400 lx 1000 lx

Ongoing research on readability

1. Influence of texturization 1. Influence of texturization 1. Influence of texturization 1. Influence of texturization on on on on HWDs:HWDs:HWDs:HWDs:

• Classify industrial backgrounds

• Validation (user test)

• Methods to improve readability

2. Readability with projection2. Readability with projection2. Readability with projection2. Readability with projection

• Projection vs. LCD

• Industrial workbenches (500 lx)

• Industrial colors

AuthoringInterfaceTrackingReadability

2. Tracking

AuthoringInterfaceTrackingReadability

Traditional

approach

3D vs. 2D tracking

CAD Database

Algorithms

2D

Algorithms

3D

2D video

streamingPoint cloud

+ video

streaming

mesh

Depth cameraVideo camera

matching

2D Marker

Databasematching

AuthoringInterfaceTrackingReadability

In collaboration

with VRAC– Iowa

State University

3. User Interface

AuthoringInterfaceTrackingReadability

Interface navigation

GoalGoalGoalGoal

• Use Kinect for gesture recognition to

navigate AR manuals

IssuesIssuesIssuesIssues

• Intuitive gestures (no training)

• Industrial compatibility (e.g. tools)

• Robust (no false positives)

• Ergonomics (RULA and Nielsen’s

method)

AuthoringInterfaceTrackingReadability

4. AR authoring

AuthoringInterfaceTrackingReadability

AR authoring

IssuesIssuesIssuesIssues

• AR authoring is time consuming

• NO direct path from CAD to AR maintenance

• Literature does not report real industrial workflows

MethodsMethodsMethodsMethods

• Open manual standard (openmanual.org)

• Leverage HTML technologies (e. g. screen resolution, text

formatting, device, etc.)

AuthoringInterfaceTrackingReadability

CADGraphic tools

(e.g. 3ds Max)

AR

application

Workflow

• AR extended open manual XML standard

• A compiler creates the AR manual

Open manual standard

(AR extension)

Compiler

AR Manual

CAD

PLM

AuthoringInterfaceTrackingReadability