Mobile AR for Urban Design

Using Mobile Augmented Reality for Urban Design

Mark Billinghurst, Gun Lee HIT Lab NZ

University of Canterbury

September 2012

Christchurch Before and After

  3D graphics solutions available   Autodesk REVIST, ESRI ArcGIS, Grass,etc

Emerging Novel User Interfaces   Immersive visualization   Multi-touch screens   Gesture interaction   Etc..

Limitations   Interface

 Complex to use  Unintuitive interaction   2D display for 3D content

  Context of Use   Remote visualization   Separation from the real world  Unable to show life-sized content

1983 – Star Wars

Augmented Reality  Combines Real and Virtual Images

-  Both can be seen at the same time   Interactive in real-time

- The virtual content can be interacted with  Registered in 3D

- Virtual objects appear fixed in space

AR for Urban Design

MagicCup: Urban Design   Intuitive Virtual Object Manipulation

on a Table-Top Workspace

 Design your own city   Tangible User Interface

-  Intuitive Manipulation

Mobile AR for Urban Design   Enhancing plans/documents

  Indoor AR use, meeting support  Overlaying virtual content on physical objects

-  Visual tracking

  Showing content on-site  Outdoor AR use, on-site visualization  Geo-located points of interest (POI)

-  Sensor based tracking (GPS, compass)

Enhanced City Plans

  CERA – CCDU Plan  Using tablet to track off printed maps  Overlay 3D city models onto real maps

User Experience

  Touch interaction   Pointing selection   Animated objects

Outdoor AR (2003)

  Backpack system   Laptop, HMD, camera

  Highly accurate tracking  GPS, inertial, RTK system   2-3 cm accuracy

User Views

  Stakeout survey application  Users able to find survey points twice as fast

2008 - Location Aware Phones

Nokia Navigator Motorola Droid

Real World Information Overlay   Tag real world locations (POI)

 GPS, compass input  Overlay graphics data on live video

  Applications   Travel guide, Advertising, etc

  Eg: Layar (www.layar.com)   > 20 million users   > 3,000 information channels

3DOn

  Onsite Visualization Single Building  GPS + Compass input  Overlay graphics data on live video/Photos

HIT Lab NZ Outdoor AR Platform

  Cross platform   Android, iPhone

  3D onsite visualization   3D model loading   Intuitive user interface

  Positions content in space  Camera, GPS, compass

  Targeting museum guide/outdoor site applications

CityViewAR Application (2011)

  Visualize Christchurch before the earthquakes

User Experience

  Multiple Views  Map View, AR View, List View

  Multiple Data Types   2D images, 3D content, text, panoramas

Time Usage

  Percentage time spent in different views   AR used more then 50% of time when available

Navigation Patterns

  Red: AR users   Blue: Non-AR users

Survey Results   Use of AR improved the user experience

  But no difference between AR and non-AR usefulness

  Favourite features   AR and Panorama views most popular  Users also enjoyed having rich data available

  Main Problems (50% reported no problems)  UI design not intuitive (24%)   System not responsive (16%)

AR Urban Design Tool (2012)   CERA CCDU Application

  Enhanced CityViewAR application

  Add VR view mode   3D model viewing/interaction

  Add concept models of buildings  Multiple buildings at single site

  Client/Server architecture   Support for user generated feedback

VR View

Differing Views

  List View, Content View, Map View

Interaction   Touch interaction

  Information filtering   Building information   Model panning, zoom, rotation

Looking to the Future

Next Steps   Subsurface Visualization

  See underground infrastructure

  User evaluation   Architects, Urban Designers, Public

  Develop authoring tool  Web-based content management system

  Improve technology   Tracking, 3D rendering

Subsurface Visualization

  Using AR to view underground infrastructure

Client/Server Architecture

Android application

Web application java and php server

Database server Postgres

Web Interface

Add models

Web based Outdoor AR Server   Web interface

  Showing POIs as Icons on Google Map

  PHP based REST API  XML based scene

data retrieval API   Scene creation and

modification API   Android client side

REST API interface

BASIC VIEW

PERSONAL VIEW

Augmented Reality 2.0 Infrastructure

Leveraging Web 2.0   Content retrieval using HTTP   XML encoded meta information

  KML placemarks + extensions

  Queries   Based on location (from GPS, image recognition)   Based on situation (barcode markers)

  Syndication   Community servers for end-user content   Tagging

  AR client subscribes to data feeds

iceFest (2012)

  Outdoor AR for Antarctic Experience   Visit Antarctica in real world   Using handheld tablet

Map Views

Content Views

AR View

  View of ice superimposed over real world

Panorama

  Immersive 360 panorama

Feedback

  A = Satisfying, B = Ease to use   C = Useful for learning, D = Better learning than internet

Conclusions   AR provides for geo-spatial visualization in place

  Text, Images, 3D models

  Hardware and software platforms widely available   Many possibilities for urban design applications

  Indoor, outdoor AR enhancements

  Important research problems need to be solved   Improved tracking, user interface, authoring tools,

social interaction, user experience, etc.

More Information   Mark Billinghurst

 [email protected]

  Gun Lee   [email protected]

  Websites  www.hitlabnz.org  www.hitlabnz.org/cityviewar  www.hitlabnz.org/mobileAR

Try It Yourself   iOS (iPhone/Android)

  Search for CCDU on app store

  Android – Search on Android Market  CityViewAR – outdoor AR view  CCDU 3D – outdoor AR and 3D view  CCDU AR – AR view

Thanks   People

  Julian Looser   Jason Mills

  Organizations