1 Panoramic University of Amsterdam Informatics Institute
Dec 18, 2015
4
RoboCup 4-Legged League
Sony Aibo Robots
4 vs. 4 robots play fully autonomously
Soccer Games
Con
text
: M
obile
Rob
ots
5
RoboCup @ home
real-world applications human-machine interaction
Fully autonomous robots have to master challenges in unknown & unstructured environments
Follow a human, navigate, etc.
7
Traditional approaches
• Aibos / 4-Legged league uses landmarks with
known positions,known shape andknown color (manually calibration taking
hours)
• General solutions (SLAM) use better hardware
• Laser range finders• Omnidirectional cameras• Robots with better odometry (wheels)
The
pro
blem
:M
obile
rob
ot lo
caliz
atio
n(e
stim
atin
g th
e ro
bot’s
pos
ition
)
8
Features of new approach
• Real-time localization on a Sony Aibo• Take advantage of natural features of a
room– Independency of artificial landmarks– Auto-calibrating in new environments
• Idea:– Learn a panoramic model of the surroundings
of the robot for localization
9
Color clustering
Collect interesting colors
(around the robot)
Determine 10 most characteristic colors(using an EM clustering
algorithm)
Raw image(208x160,
YCbCr)
10
Sector appearance
Divide in vertical slices, called sectors
(360° correspond to 80 sectors)
Count color transitionsper sector
(between the 10 most char-acteristic colors of the scene)
App
roac
h B
uild
ing
an v
irtua
l pan
oram
ic w
all
Raw image(208x160,
YCbCr)
11
Learning the panorama model
Image features(10-12 sectors/image,
10x10 frequencies/sector)
Learn panorama model(estimate frequency distributions
per sector)
Panorama model(80 sectors,
10x10 distributions, each defined by 5 bins)
12
Alignment and Localization
Image features(10-12 sectors/image,
10x10 frequencies/sector)
Align with stored panorama model
(find shortest path)
Output(Rotational estimateSignal-to-noise ratioConfidence range)
After learning from 131 frames Robot rotated 45° to the left
13
Experiments in human environments
• Rotational test in living room (at night)
Res
ults
L
earn
ing
of th
e ap
pear
ance
of u
nkno
wn
& u
nstr
uctu
red
envi
ronm
ents
14
4-Legged soccer field, indoors,single learned spot
Translational test on soccer fields
Human soccer field, outdoors,single learned spot
15
Multi-spot learning
• Aibo trained on 4 different spots, yielding 4 different panoramas
• Aibo kidnapped and placed back at arbitrary positions on the field
• Aibo tries to walk back to center spot
16
Possibilities for the 4-Legged league
• Getting rid of all artificial landmarks
• 11 vs. 11 games (bigger field)
• Outdoor demonstrations become possible
Con
clus
ions
17
Possible usage for theRoboCup @ home league
• Distinguish living room from kitchen or garden
• Rough but quick map building
• Find relative position of the TV/stove/etc on this map
18
Other applications
• CareBot: navigation in a closed indoor environment
• Mobile applications (for example on cellular phones) for quick positional estimates (tourism)
19
Rotational estimate and Confidence range in numbers
0
22.5
45
67.5
90
0 50 100 150 200 250
Distance from learned spot (mm)
de
gre
es
confidence range
error in rotation estimate
20
Architecture Overview
Camera images
Odometry updatesd
Sector-based feature
extraction
and SNR
Align with panorama model
0º
90º
180º
270º
Orientation buffer
+ confidence range
.. ..
21
Conclusions
• Accurate estimate of the rotation from a single learned spot (up to 40 meters)
• A good estimate of the relative distance from a single learned spot (up to 40 meters)
• Rough estimate of the absolute position from multiple trained spots
22
University of AmsterdamInformatics Institute
Panoramic Localization with a Sony Aiboby Jürgen Sturm
User manual• Head button always resets robot and triggers autoshutter & color clustering• Press front button to manually trigger color clustering
In training mode:• Press middle button to start learning of the first spot• Press middle button again to continue learning on more spots• Press back button to switch to localization mode
In localization mode:• Press front button to switch between rotational and translational mode• Press middle button to reset panorama and start learning• Press back button to switch between find and set-reference mode
1
2
1 5
3
1
4
7
8 9
6
Fullly working memorystick image can be downloaded fromhttp://staff.science.uva.nl/~jsturm/panorama/panorama-release.zip