Dana Cobzas-PhD thesis Image-Based Models with Applications in Robot Navigation Dana Cobzas Supervisor: Hong Zhang.

Dana Cobzas-PhD thesis

Image-Based Modelswith Applications inRobot Navigation

Dana CobzasSupervisor: Hong Zhang

Dana Cobzas-PhD thesis

3D Modeling in Computer Graphics

Graphics model: 3D detailed geometric model of a scene

Goal: rendering new views

New viewReal scene Geometric model + texture

RenderingAquisition

Range sensors

Modelers

[Pollefeys & van Gool]

Dana Cobzas-PhD thesis

Mapping in Mobile Robotics

Robot Map

sensors

Map Building

Localization/TrackingNavigation

environment

Navigation map: representation of the navigation space

Goal: tracking/localizing the robot

Dana Cobzas-PhD thesis

Same objective: How to model existing scenes?

Traditional geometry-based approaches:

= geometric model + surface model + light model- Modeling complex real scenes is slow - Achieving photorealism is difficult- Rendering cost is related to scene complexity+ Easy to combine with traditional graphics

Alternative approach: image-based modeling: = non-geometric model from images

- Difficult to acquire real scenes- Difficult to integrate with traditional graphics+ Achieving photo-realism is easier if starting from real photos+ Rendering cost is independent on scene complexity

In this work we combine the advantages of both for mobile robotics localization and predictive display

Dana Cobzas-PhD thesis

This thesisInvestigates the applicability of IBMR techniques in mobile robotics.

Questions addressed: Is it possible to use an IBM as navigation map for

mobile robotics? Do they provide desired accuracy for the specific

applications – localization and tracking? What advantages do they offer compared to traditional

geometric-based models?

Dana Cobzas-PhD thesis

Approach

Solution: Reconstructed geometric model combined with

image information 2 models

Model1: calibrated: panorama with depthModel2: uncalibrated: geometric model with

dynamic texture Applications in localization/tracking and

predictive display

Dana Cobzas-PhD thesis

Model1: Panoramic model

Dana Cobzas-PhD thesis

Model1: Overview

Standard panorama: - no parallax, reprojection from the same viewpoint

Solution – adding depth/disparity information:1. Using two panoramic images for stereo

2. Depth from standard planar image stereo

3. Depth from laser range-finder

Dana Cobzas-PhD thesis

Depth from stereo

Cylindrical image-based panoramic models

+ depth map

Trinocular Vision System

(Point Gray Research)

Dana Cobzas-PhD thesis

Depth from laser range-finder

180 degrees panoramic mosaic

Corresponding range data (spherical representation)

Data from different sensors: requires data registration

CCD camera Laser rangefinder Pan unit

Dana Cobzas-PhD thesis

Model 1: ApplicationsAbsolute localization:

Input: image+depthFeatures: planar patches

vertical lines

Input: intensity imageAssumes: approximate poseFeatures: vertical lines

Incremental localization:

Predictive display:

Dana Cobzas-PhD thesis

Model 2:Geometric model with dynamic

texture

Dana Cobzas-PhD thesis

Model 2: Overview

Input images Model Applications

Geometric model

Dynamic texture

Tracking

Rendering

Dana Cobzas-PhD thesis

Tracked features

Structure from motion

algorithm

poses

structure

Geometric structure

Dana Cobzas-PhD thesis

Dynamic texture

I1

tI

Input Images Re-projected geometry

Texture Variability basis

Dana Cobzas-PhD thesis

3D SSD Tracking Goal: determine camera motion (rot+transl) from image

differences Assumes: sparse geometric model of the scene Features: planar patches

past motion

currentmotion

past warp

current warpdifferential

warp

differentialmotion initial

motion

3D Model

Dana Cobzas-PhD thesis

Tracking example

Dana Cobzas-PhD thesis

Tracking and predictive display

Goal: track robot 3D pose along a trajectory Input: geometric model (acquired from images) and initial

pose Features: planar patches

Dana Cobzas-PhD thesis

Thesis contributionsContrast calibrated and uncalibrated methods for

capturing scene geometry and appearance from images:

panoramic model with depth data (calibrated)

geometric model with dynamic texture (uncalibrated)

Demonstrate the use of the models as navigation maps with applications in mobile robotics

absolute localizationincremental localizationmodel-based trackingpredictive display

Dana Cobzas-PhD thesis

Thesis questions

What advantages do they offer compared to traditional geometric based models? The image information is used to solve data association

problem. Model renderings are used for predicting robot location for a

remote user.

Do they provide desired accuracy for the specific applications – localization, tracking? The geometric model (reconstructed from images) is used

for localization/tracking algorithms. The accuracy of the algorithm depends on the accuracy of the reconstructed model.

The model accuracy can also be improved during navigation as different levels of accuracy are needed depending on the location (large space/narrow space) – future work.

Is it possible to use an image-based model as navigation map for mobile robotics? A combination of geometric and image-based model can be

used as navigation map.

Dana Cobzas-PhD thesis

Comparison with current approaches

Mobile Robotics Map+ Image information for data association+ Complete model that can be rendered – closer to human

perception- Concurrent localization and matching (SLAM-Durrant-Whyte)- Invariant features (light, occlusion) (SIFT-Lowe)- Uncertainty in feature location (localization algorithms)

Graphics Model (dynamic texture model-hybrid image+geometric model)

+ Easy acquisition: non-calibrated camera (raysets, geometric models)

+ Photorealism (geometric models)+ Traditional rendering using the geometric model (raysets)- Automatic feature detection for tracking – larger scenes- Denser geometric model (relief texture)- Light-invariance (geometric models, photogrammetry)

Dana Cobzas-PhD thesis

Future workMobile Robotics Map

Improve map during navigation Different ‘map resolutions’ depending on robot pose Incorporate uncertainty in robot pose and features Light, occlusion invariant features Predictive display: control robot’s motion by ‘pointing’ o

‘dragging’ in image space

Graphics Model (dynamic texture) Automatic feature detection for tracking Light-invariant model Compose multiple models into a scene based on intuitive

geometric constraints Detailed geometry (range information from images)