1 Augmented and Mixed Reality Applications at VTT Charles Woodward April 16, 2010 VTT TECHNICAL RESEARCH CENTRE OF FINLAND Mobile Augmented Reality one of the technologies to “most likely to alter industries, fields of research, and the way we live”. - MIT’s annual review 10 Emerging Technologies 2007 - Gartner’s Top Ten Disruptive Technologies 2008-2012 - First consumer applications on camera phones 2009 - Commercial breakthrough predicted 2010
23
Embed
Augmented and Mixed Reality - oamk.fi · Project “AR4BC” Augmented Reality for Building and Construction ... Augmented Reality + Augmented Virtuality ... CATIA -> STEP -> Reduced
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Augmented and Mixed Reality
Applications at VTT
Charles Woodward
April 16, 2010
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Mobile Augmented Reality
one of the technologies to
“most likely to alter industries, fields of research,and the way we live”.
General VR/AR expertise around VTT14 research teams in VR/AR roadmap workApplications from games to digital media,construction to production, ships, cars, military, …Technology incl. core VR/AR solutions, sensors,GSM/GPS positioning, multimodality, 3D audio,motion tracking, ubi & ambient intelligence,Special strengths: Multi-disciplinary research,strong industry relations and experience
Related technologiesAugmented VirtualityMediated RealityDiminished Reality
-> Mixed Reality (MR)
4
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
ARCatalogueViewing with data glasses / web cameraStereo 3D products appear from catalogueBased on marker detection from camera view
Applications, e.g.B2B marketing (by Lappset Group )3D viewing of engineering drawings (by Pöyry Inc.)
Also, camera phonesConsumer solutions
Example Application
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
3D animated chirdren’s TV seriesBy Futurecode (Finland) and Blue Arc (China)Launched April 2nd, 2010, MTV3 / SubTV Junior
Augmented magazine advertisementTV-magazines Katso and TVSeiska, March 20103D animations appear augmented from magazineInteraction with augmented Dibidog
Dibidogs
5
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Mobile phone implementationIntroduced at Sanghai World Expo, May 1st, 2010Nokia Maemo (N900) and Symbian S60 platforms
Augmented childrens’ storybookKustannus Paasilinna, August 2010Various new interactions and narrativesMarkerless tracking, photorealistic rendering etc.
Dibidogs, Next Steps
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
ALVAR Subroutine Library
”A Library for Virtual and Augmented Reality””AR engine” developed by VTTFree trial download at www.vtt.fi/multimedia/alvar.htmlUsers e.g. Columbia Univ. (Goblin XNA), MIT (6th Sense)
Detecting and tracking 2D markersAccurate marker pose estimation, with erro concealing“Marker fields” can be defined manually, or autocalibratedHiding of markers as sepcial feature
UtilitiesCamera calibration; Correcting lense undistortions; Opticalflow tracking; Several basic filters and Kalman library;Finding exterior orientation of point-sets
Augmented networked table tennisPlayed with real rackets, color markerVideo image of opponent shown at other endof tableVirtual ball, table and net
FeaturesBall speed, racket size can be adjustedSolo game against a wall, or against "oneself"Game viewing by Internet audienceRecording of game
BenefitsSocially and physically activatingNatural human-computer user interactionFits in small interiors, at home or officeSuits also for kids and elderly peopleNo special hardware required
7
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Mobile Entertainment - SymBall
Virtual table tennis game for camera phonesCamera driven game control by feature/motion detectionData connection between players via Bluetooth or GPRSFirst networked camera driven game in the world
ImplementationSymbian OS/Series 60 (& J2ME)Features: sounds, ball speed, etc.3D stereo game using r/b goggles!
Future plan: Augmented versionUsing turnable camera, face detectionShow real opponent on phone display
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Internet
Experimental Work ”TelePet”Augmented Networked Games for Dogs
To entertain dogs over Internet during master’s absenceGame control based on gesture detection
Example gameMaster throws virtual ball, dog has to “catch”Rewards, e.g. biscuit box operated over internetInitial animal tests - positive results
8
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Vappunaamari.comVirtual masks for May Day party
Face detection/augmenting on camera phonesProvided as public service, > 1500 downloadsFeatures: send photo to friends, user created masks,compressing augmented video, …
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
INTERIOR DESIGN
9
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Augmented Interior Design – Alternative Solutions
Basic solutionVirtual furniture object tied to eachmarkerInterior designed by moving markerson the floorDisplay on PC screen or data glasses
BenefitsFurniture is ”light” to move aroundEasy to try out different alternativesNo modelling of real environmentrequired
Things to improve …Web camera … isn’t very mobileData glasses / PC .. not tooconvenient
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Augmenting in Digital Camera Images
Application ”ARInteriors”Single marker placed on floor, definingscale and perspectiveDigital camera images of the room areuploaded to computerVirtual furniture is selected and placedwith menu/mouseFeatures: lighting, shadows, hiding ofmarker etc.
BenefitsHardware (camera, PC) found inevery homeBetter image quality, resolution,accuracy etc.Photorealistic 3D rendering
10
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Augmenting on TabletPC
Application “ARMobile”Tablet PC (Sony Vaio), cameraintegratedOptional data glassesUser interface to fit small screen,stylus inputNo configuration files requiredArbitrary marker orientationsMarker fields calculated automaticallyMoving, scaling and rotating objectsAdjusting of lights, with soft shadowsErasing of markersStoring of video clip & projectKalman filter & other stabilisationFeature based tracking when markersdisappear
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Find appealingfurniture in ad
Download virtualfurniture viamatrix code
Instantly viewfurniture at homeon mobile phone
Verify by high-endPC solution (ifrequired)
Get to shop,purchase product
Augmenting on Mobile Phones
Application ”ARPhone”Placing virtual furniture on phone’s camera imageModels are loaded using matrix code in product catalogue or advertUser places the furniture in camera view by 3D rotation and zoomFurniture stays in place when moving the camera phoneBased on feature tracking - no markers requiredAlso, augmenting of real furniture
11
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
ARStudio
Unified solution for general AR applicationsFormats: real time / stored video <-> still imagesSingle/multimarker tracking -> markerless methodsMatching augmented content with video qualityPhotorealism, shadows, materials, lighting, …
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
VividAR
AR interior design solution by VividWorksBased on VTT’s technologyLaunched February 2010Consumer web solution later this spring
12
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
ARCHITECTURE AND CONSTRUCTION
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Case Study: VTT’s Digitalo officesDigitalo = “DigiBuilding”Construction started Dec 2003Building ready Oct 2005
Various AR/VR methods usedRadiosity visualization3D LandscapesCAVE Virtual RealityAugmented Scale ModelAugmented Web CameraMobile Augmented Reality
Projects involvedCommissions by Senate PropertiesProject ARARAT 2004-2006
Augmented Reality in Architecture
13
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Digitalo augmented on conference tableViewing through video glasses (stereo)Move around model, communicate with people
FeaturesVirtual menu for user interactionVirtual reality tour utilising feature based trackingPredefined walk-through routes, with hyperlinksRealism: textures, reflections, shadows etc.Add, move and remove parts (AR-catalogue)
Augmented Scale Model
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Digitalo augmented in live web camera viewPresentation for future users of buildingFollow-up and comparison with original plansView with or without virtual buildingReal-time interaction (zoom, pan, etc.)Viewing inside to the building, etc.
Ongoing development (project AR4BC)Integrating 4D IFC models to live web cameras
Augmented Web Camera
14
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Outdoors visualization of DigitaloPioneering work by VTT 2003
Remote Maintenance TasksITER fusion reactor mock-up at TampereMaintenance robot augmented / animatedinside test platformApplication e.g. for discrepancy checking(after accident or other changes)
Implementation phases3D Puzzle, multimodal interactionCamera phone implementation, client/serverIndustrial case: hydraulic block by Valtra PlcIncl. data pipeline CAD -> AR, user studies etc.
19
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Data Pipeline
Data pipeline from 3D CAD to augmented assemblyCATIA -> STEP -> Reduced B-rep -> Design for Assembly -> Animations -> Augmenting
Production parameters
and assemblability fig
ures
Assembly structure
Production parameters
and assemblability fig
ures
Assembly structure
Work Phase:23
Return tube
Tool:27 mm wrench
Work Phase:23
Return tube
Tool:27 mm wrench
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Client/servereasy to implement, general solutiontracking and other heavy computation can be done on serverno need to store large model data on mobile deviceusing subimages, no need for any 3D support on mobile device
Still imagesin most cases, the assembly station is staticit is sufficient that the augmented content is “live”it is more ergonomic to view the image without having to point at targetrobust, provides good image size (accuracy) for trackingenables general implementation, e.g. on Java ME
Assembling is on average quicker with animatedinstructions and results in less mistakes than with paperinstructions.The usage of wrong tool and false positioning are onaverage less frequent with augmented-reality instructionsthan with paper ones.A target for improvement is the HMD hardware. Operationwith the helmet and wires was often experienceduncomfortable.
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Project ”ManuVAR”
”Manual Work Support Throughout SystemLifecycle by VR/AR”
EU FP7 NMP “Beyond lean manufacturing”18 partners, 70+ participants, VTT as coordinatorDuration 2009-2012, budget 9.7M€
The general objective is to develop• (1) technology platform and• (2) methodological framework
for supporting all aspects of manual work throughout thesystem lifecycle with the ultimate goal to
1. increase productivity and quality and reduce costof manual work in the whole lifecycle
2. facilitate adaptation to product customization andchanges
3.support efficient knowledge and skillmanagement through the lifecycle
21
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Satellite assembly Remote training Manufacturing design
Maintenance ofheavy machinery
Industrial plantmaintenance
Thales Alenia Space NEM Solutions TNO
Tecnatom
Metso Minerals
ManuVAR Applications
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
RECENT PROJECTS: MIXED REALITY
22
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Mobile Mixed RealityCo-creation project “MMR” with Nokia
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Mixed Reality Video ConferenceTeleconference btw real people and SL avatars
Avatars appear augmented in real room
People’s gestures are replicated by avatars
Collaboration with virtual objects
Partners VTT, TKK, Nokia, IBM
23
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Multitouch InteractionMultitouch table developed by VTT
LCD display with cameras in the corners
Any flat surface can be used for multitouch interaction
Application example: connecting floor plan with CAVE,navigation in 3D architectural model
At Sanghai World Expo 2010: multitouch in the air,to control giant projector walls, tens of people
Collaborative work,cf. “Minority Report”
VTT TECHNICAL RESEARCH CENTRE OF FINLAND
Thank You!
Dr. Charles WoodwardVTT Technical Research Centre of Finland