Contact: Mobile Robotics and Olfaction Lab, Assoc. Prof ...130.243.105.49/Research/Learning/publications/2011/...Mining Vehicles (Atlas Copco, Fotonic) Autonomous Work Machines ...

Mobile Robotics and Olfaction Lab, AASS, Örebro University

# 1

Contact: Assoc. Prof. Dr. Achim Lilienthal

www.aass.oru.se/~lilien [email protected]

# 5 © A. J. Lilienthal (Nov 30, 2011)

1. Profile

2. Research

3. Projects

Content

# 6 © A. J. Lilienthal (Nov 30, 2011)

1. Profile

2. Research

3. Projects

Content

# 9 © A. J. Lilienthal (Nov 30, 2011)

1. Profile

2. Research

3. Projects

4. Publications

5. Future Work

Content

# 10 © A. J. Lilienthal (Nov 30, 2011)

Profile

1

# 11 © A. J. Lilienthal (Nov 30, 2011)

1.

General Focus ... o perception systems for mobile robots

(fundamentals for autonomous and safe operation)

Objective ... o advance theoretical and practical foundations that allow mobile

robots to operate in an unconstrained, dynamic environment

Approaches are Characterized by ... o fusion of different sensor modalities o timely integration into industrial demonstrators

MR&O Lab Profile

# 12 © A. J. Lilienthal (Nov 30, 2011)

1.

D1 – Mobile Robotics o for autonomous and safe long-term operation in the real world o technology transfer through collaborative projects with industrial

partners in the area of logistics robots o examples: autonomous forklifts and autonomous wheel loaders

MR&O Lab Profile – Two Major Research Directions

# 13 © A. J. Lilienthal (Nov 30, 2011)

1.

Forklift Trucks (Danaher Motion, Linde MH, Stora Enso)

Autonomous Work Machines

speed x 2

# 14 © A. J. Lilienthal (Nov 30, 2011)

1.

Forklift Trucks (Danaher Motion, Linde MH, Stora Enso)

Wheel Loaders (VolvoCE, VolvoTech, NCC)

Autonomous Work Machines

# 15 © A. J. Lilienthal (Nov 30, 2011)

1.

Forklift Trucks (Danaher Motion, Linde MH, Stora Enso)

Wheel Loaders (VolvoCE, VolvoTech, NCC)

Mining Vehicles (Atlas Copco, Fotonic)

Autonomous Work Machines

picture from the actual mine!

# 16 © A. J. Lilienthal (Nov 30, 2011)

1.

Forklift Trucks (Danaher Motion, Linde MH, Stora Enso)

Wheel Loaders (VolvoCE, VolvoTech, NCC)

Mining Vehicles (Atlas Copco, Fotonic)

Hospital Transport Vehicles (RobCab)

Autonomous Work Machines

# 20 © A. J. Lilienthal (Nov 30, 2011)

1.

Forklift Trucks (Danaher Motion, Linde MH, Stora Enso)

Wheel Loaders (VolvoCE, VolvoTech, NCC)

Mining Vehicles (Atlas Copco, Fotonic)

Hospital Transport Vehicles (RobCab)

Garbage Bin Collection and Cleaning (RoboTech)

Mobile Work Machines

# 21 © A. J. Lilienthal (Nov 30, 2011)

1.

DustBot

Mobile Work Machines

# 23 © A. J. Lilienthal (Nov 30, 2011)

1.

D2 – Artificial and Mobile Robot Olfaction o Artificial Olfaction = gas sensing with artificial sensor systems o we study particularly open sampling systems def o develop "electronic nose" towards a "mobile nose" o examples: gas sensor networks (air pollution monitoring), mobile

robots for surveillance of landfill sites, gas leak localization

MR&O Lab Profile – Two Major Research Directions

# 24 © A. J. Lilienthal (Nov 30, 2011)

1.

Forklift Trucks (Danaher Motion, Linde MH, Stora Enso)

Wheel Loaders (VolvoCE, VolvoTech, NCC)

Mining Vehicles (Atlas Copco, Fotonic)

Hospital Transport Vehicles (RobCab)

Garbage Bin Collection and Cleaning (RoboTech)

Landfill Site Inspection (Atleverket)

Mobile Work Machines

# 26 © A. J. Lilienthal (Nov 30, 2011)

1.

Senior Staff (12) o Assoc. Prof. Dr. Achim J. Lilienthal o Dr. Henrik Andreasson o Dr. Erik Berglund o Dr. Abdelbaki Bouguerra o Dr. Marcello Cirillo o Dr. Dimitar Dimitrov o Dr. Boyko Iliev o Dr. Martin Magnusson o Dr. Marco Trincavelli o Assoc. Prof. Dr. Anani Ananiev o Adj. Prof. Dr. Rainer Palm o Prof. Dr. Ivan Kalaykov

Guests (2) o Rafael Mosberger o Alejandro Rituerto Sin

MR&O Lab Profile – People

PhD Students (15) o Houssam Albitar o Mohamad Aldanmad o Håkan Almqvist o Sahar Asadi o Victor Hernandez Bennets o Krzysztof Charusta o Kinan Dandan o Robert Krug o Rasoul Mojtahedzadeh o Patrick Neumann (assoc.)

o Karol Niechwiadowicz o Sepideh Pashami o Matteo Reggente o Morgan Rody o Todor Stoyanov

# 27 © A. J. Lilienthal (Nov 30, 2011)

1.

Funding o 19 people 11.4 MSEK (approx. 1.25M€) in 2011

» external/internal index: 8.52

MR&O Lab Funding, 2011

# 29 © A. J. Lilienthal (Nov 30, 2011)

Research

2

# 30 © A. J. Lilienthal (Nov 30, 2011)

2.

Major Research Areas we are internationally recognized for work in ... o rich 3D perception o robot vision o mobile robot olfaction

MR&O Lab, Research Areas

# 32 © A. J. Lilienthal (Nov 30, 2011)

2.

Major Research Areas we are internationally recognized for work in ... o rich 3D perception o robot vision o mobile robot olfaction

MR&O Lab, Research Areas

# 33 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o rich 3D = spatial data augmented with additional information

» additional dimensions: e.g. colour, temperature, semantic information, …

MR&O Lab, Research Areas

coloured point cloud (an example of rich 3D data)

# 34 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data?

MR&O Lab, Research Areas

# 36 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» how to create compact, consistent world models from 3D data? • 3D-NDT (Normal Distribution Transform) [Magnusson et al., JFR 2007]

MR&O Lab, Research Areas

# 37 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» how to create compact, consistent world models from rich 3D data? » scan registration def

MR&O Lab, Research Areas

# 38 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» how to create compact, consistent world models from rich 3D data? » scan registration def

• Iterative 3D-NDT Scan Registration [Magnusson et al., JFR 2007 / Magnusson et al., ICRA 2009]

MR&O Lab, Research Areas

3D-ICP

3D-NDT

point-to-NDT

# 41 © A. J. Lilienthal (Nov 30, 2011)

2.

[Magnusson et al., JFR 2007] [Magnusson et al., ICRA 2009]

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» how to create compact, consistent world models from rich 3D data? » scan registration def

• Iterative 3D-NDT Scan Registration [Magnusson et al., JFR 2007 / Magnusson et al., ICRA 2009]

• NDT-2-NDT Registration [Stoyanov et al., ICRA 2012? / IJRR 2012?]

point-to-NDT (P2D)

[Magnusson et al., JFR 2007] [Magnusson et al., ICRA 2009]

MR&O Lab, Research Areas

point-to-NDT (P2D)

[Stoyanov et al., IJRR 2012?] [Stoyanov et al., ICRA 2012?]

NDT-to-NDT (D2D)

# 42 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

NDT-to-NDT (D2D)

ICP

[Besl/McKay, TPAM 1992]

# 43 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

NDT-to-NDT (D2D)

ICP Generalized ICP

[Besl/McKay, TPAM 1992] [Segal et al., RSS 2009]

# 44 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

NDT-to-NDT (D2D)

ICP Generalized ICP Point-to-NDT (P2D)

[Besl/McKay, TPAM 1992] [Segal et al., RSS 2009] [Magnusson et al., JFR 2007] [Magnusson et al., ICRA 2009]

# 45 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

NDT-to-NDT (D2D)

ICP Generalized ICP Point-to-NDT (P2D) NDT-to-NDT (D2D-L2)

[Besl/McKay, TPAM 1992] [Segal et al., RSS 2009] [Magnusson et al., JFR 2007] [Magnusson et al., ICRA 2009]

[Stoyanov et al., IJRR 2012?] [Stoyanov et al., ICRA 2012?]

# 46 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

» results

NDT-to-NDT (D2D)

ICP

343 registrations

# 47 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

» results

NDT-to-NDT (D2D)

ICP NDT-P2D

343 registrations

# 48 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

» results

NDT-to-NDT (D2D)

ICP NDT-P2D NDT-D2D (L2)

343 registrations

# 49 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

» results

NDT-to-NDT (D2D)

ICP NDT-P2D NDT-D2D (L2)

343 registrations

# 50 © A. J. Lilienthal (Nov 30, 2011)

2.

Scan Registration o NDT-2-NDT [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

» results

NDT-to-NDT (D2D)

ICP NDT-P2D NDT-D2D (L2)

343 registrations

good convergence, accurate and consistent,

slow

very good convergence, very accurate and

consistent, very fast

varying convergence, consistent but least

accurate, slow

# 53 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» how to create compact, consistent world models from rich 3D data? » scan registration def

• Iterative 3D-NDT Scan Registration [Magnusson et al., JFR 2007 / Magnusson et al., ICRA 2009]

• NDT-2-NDT Registration [Stoyanov et al., ICRA 2012? / Stoyanov et al., IJRR 2012?]

• Registration using Depth-Interpolated Local Image Features [Andreasson/Lilienthal, RAS 2010]

MR&O Lab, Research Areas

# 54 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» how to detect changes in rich 3D data? ("Find the Difference") • Difference Detection for Security Patrol Robots [Andreasson et al., IROS 2007]

MR&O Lab, Research Areas

# 55 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» detecting re-visited places with a low-dimensional representation? • 3D-NDT Signatures for Loop Closing [Magnusson et al., JFR 2009 / ICRA 2009]

MR&O Lab, Research Areas

# 56 © A. J. Lilienthal (Nov 30, 2011)

2.

Rich 3D Perception o research question

» how can we create a consistent, useful world model from rich 3D data? o key contributions

» can we plan paths directly in a compact representation? • NDT-based Path Planning in 3D Environments [Stoyanov/Lilienthal IROS 2010]

MR&O Lab, Research Areas

# 112 © A. J. Lilienthal (Nov 30, 2011)

Future Work

5

# 113 © A. J. Lilienthal (Nov 30, 2011)

5.

Perception – Open Research Problems o dynamic environment o 3D, rich 3D (flat floor assumption does not hold)

» large amount of data, compact yet versatile representation required

o online solutions » incremental updates » constrained resources

o high speeds » scanning-while-moving » constrained resources

o robust solutions » outdoor conditions » graceful degradation wrt errors

o efficiency (e.g. compared to human operator)

Outlook – Dynamic Environment

# 114 © A. J. Lilienthal (Nov 30, 2011)

5.

Toyota SLAM Benchmark 2006

Outlook – Dynamic Environment

# 117 © A. J. Lilienthal (Nov 30, 2011)

5.

Perception – Open Research Problems o dynamic environment o 3D, rich 3D (flat floor assumption does not hold)

» large amount of data, compact yet versatile representation required

o online solutions » incremental updates » constrained resources

o high speeds » scanning-while-moving » constrained resources

o robust solutions » outdoor conditions » graceful degradation wrt errors

o efficiency (e.g. compared to human operator)

Outlook – Perception for Autonomous Vehicles

Mobile Robotics and Olfaction Lab, AASS, Örebro University

# 119

Contact: Assoc. Prof. Dr. Achim Lilienthal

www.aass.oru.se/~lilien [email protected]