CV 輪講 Putting Objects in Perspective
Post on 07-Feb-2016
101 Views
Preview:
DESCRIPTION
Transcript
CV 輪講Putting Objects in
Perspective
藤吉研究室 土屋成光 2008年 7月 1日
Back ground
一般物体認識 / 画像シーン認識– 低解像度– 見えの違い– 奥行きによるサイズの違い ⇒局所的な認識法が通用しない
人間は物体間の関係を利用– 三次元構造のモデル化– 局所的な認識手法を高精度に
Putting Objects in Perspective
Derek Hoiem ， Alexei A. Efros ， Martial Hebert
Carnegie Mellon University Robotics Institute
CVPR2006
Understanding an Image
Today: Local and Independent
検出結果
Local Object Detection
True Detection
True Detections
MissedMissed
False Detections
Local Detector: [Dalal-Triggs 2005]
Object Support
Surface Estimation
Image Support Vertical Sky
V-Left V-Center V-Right V-Porous V-Solid
[Hoiem, Efros, Hebert ICCV 2005]
Software available online
ObjectSurface?
Support?
Object Size in the Image
Image World
Input Image
Object Size ↔ Camera Viewpoint
Loose Viewpoint Prior
Object Size ↔ Camera Viewpoint
Input Image Loose Viewpoint Prior
Object Position/Sizes Viewpoint
Object Size ↔ Camera Viewpoint
Object Position/Sizes Viewpoint
Object Size ↔ Camera Viewpoint
Object Position/Sizes Viewpoint
Object Size ↔ Camera Viewpoint
Object Size ↔ Camera Viewpoint
Object Position/Sizes Viewpoint
Efficient from surface and viewpoint
Image
P(object) P(object | surfaces)
P(surfaces) P(viewpoint)
P(object | viewpoint)
Image
P(object | surfaces, viewpoint)
Efficient from surface and viewpoint
P(object)
P(surfaces) P(viewpoint)
Scene Parts Are All Interconnected
Objects
3D SurfacesCamera Viewpoint
Input to Algorithm
Surface Estimates Viewpoint Prior
Surfaces: [Hoiem-Efros-Hebert 2005]
Local Car Detector
Local Ped Detector
Object Detection
Local Detector: [Dalal-Triggs 2005]
Approximate Model
Objects
3D SurfacesViewpoint
s1
o1
θ
on...
sn…
Local Object Evidence
Local Surface Evidence
Local Object Evidence
Local Surface Evidence
Viewpoint
Objects
Local Surfaces
Inference over Tree
Viewpoint estimation
Viewpoint Prior
HorizonHeight Height Horizon
Like
liho
od
Like
liho
od
Viewpoint Final
Object Identitie
Local detector
Surface Geometry
Probability map
Object detection
4 TP / 2 FP
3 TP / 2 FP
4 TP / 1 FP
Ped Detection
Car Detection
Local Detector: [Dalal-Triggs 2005]4 TP / 0 FP
Car: TP / FP
Ped: TP / FP
Initial (Local) Final (Global)
Experiments on LabelMe Dataset
Testing with LabelMe dataset: 422 images– 923 Cars at least 14 pixels tall– 720 Peds at least 36 pixels tall
Each piece of evidence improves performance
Local Detector from [Murphy-Torralba-Freeman 2003]
Car Detection Pedestrian Detection
Can be used with any detector that outputs confidences
Local Detector: [Dalal-Triggs 2005] (SVM-based)
Car Detection Pedestrian Detection
Accurate Horizon Estimation
Median Error: 8.5% 4.5% 3.0%
90% Bound:
[Murphy-Torralba-
Freeman 2003]
[Dalal- Triggs 2005]
Horizon Prior
Qualitative Results
Initial: 2 TP / 3 FP Final: 7 TP / 4 FP
Local Detector from [Murphy-Torralba-Freeman 2003]
Car: TP / FP Ped: TP / FP
Qualitative Results
Local Detector from [Murphy-Torralba-Freeman 2003]
Car: TP / FP Ped: TP / FP
Initial: 1 TP / 14 FP Final: 3 TP / 5 FP
Qualitative Results
Car: TP / FP Ped: TP / FP
Local Detector from [Murphy-Torralba-Freeman 2003]
Initial: 1 TP / 23 FP Final: 0 TP / 10 FP
Qualitative Results
Local Detector from [Murphy-Torralba-Freeman 2003]
Car: TP / FP Ped: TP / FP
Initial: 0 TP / 6 FP Final: 4 TP / 3 FP
Geometric Context
Estimate surface
ground: green, sky: blue, vertical: red, o:porous, x: solid
Geometric Cues
Color
Location
Texture
Perspective
Robust Spatial Support
RGB Pixels Superpixels
[Felzenszwalb and Huttenlocher 2004]
oversegmentation
Multiple Segmentations
Superpixels
…
Multiple Segmentations
単一のセグメントではセグメントエラーの可能性 複数のセグメント数でセグメンテーション
Labeling Segments
…
…
各セグメント結果を統合
Learn from training images
前準備– multiple segmentation の算出– 各セグメントのラベルの算出 – ground, vertical,
sky, or “mixed” boosted decision trees による密度計算
– 8 nodes per tree– Logistic regression version of Adaboost
[Collins and Schapire and Singer 2002]
Label LikelihoodHomogeneity Likelihood
Image Labeling
…
Labeled Segmentations
Labeled Pixels
Learned from training images
Summary & Future Work
meters
met
ers
Ped Pe
dCar
Reasoning in 3D:• Object to object• Scene label• Object segmentation
Conclusion
Image understanding is a 3D problem– Must be solved jointly
This paper is a small step– Much remains to be done
CV 輪講Recovering Occlusion
Boundaries from a Single Image,
Closing the Loop in Scene Interpretation
藤吉研究室 土屋成光 2008年 8月 26日
Back ground
一般物体認識 / 画像シーン認識– 低解像度– 見えの違い– 奥行きによるサイズの違い ⇒局所的な認識法が通用しない
人間は物体間の関係を利用– 三次元構造のモデル化– 局所的な認識手法を高精度に
Recovering Occlusion Boundaries from a Single
Image
Derek Hoiem ， Andrew N. Stein, Alexei A. Efros ， Martial Hebert
Carnegie Mellon University Robotics Institute
ICCV’07
単画像からのオクルージョン理解
オクルージョン，境界理解– 物体を探索する際に必須– Edge, region, depth によって推定
手法の流れ
1. 千領域にセグメンテーションWatershed with Pb soft boundaries
2. Region, Boundary, 3D Cues の算出depth : horizon + junction to ground
3. Boundary の算出Conditional random field (CRF)
4. Boundary を用いて更にセグメンテーション
results
Boundary
Object popout
Closing the Loopin Scene Interpretation
Derek Hoiem ， Alexei A. Efros ， Martial Hebert
Carnegie Mellon University Robotics Institute
CVPR’08
Putting Objects in Perspective
4 TP / 2 FP
3 TP / 2 FP
4 TP / 1 FP
Ped Detection
Car Detection
Local Detector: [Dalal-Triggs 2005]4 TP / 0 FP
Car: TP / FP
Ped: TP / FP
Initial (Local) Final (Global)
Scene Parts Are All Interconnected
Objects
3D SurfacesCamera Viewpoint
with Occlusions
一般物体認識フレームワークPutting Objects in Perspective
シーン構造認識Automatic Photo Pop-up
Occlusion, Boundary 情報の利用
関係モデル
相互に関係
Putting Objects への利用
相互的に情報を利用することで高精度に
Initial : Dalal-Triggs Iter 1 : Hoiem et al. Final : This paper
Car : Up, Ped : Down群衆の境界線の精度が問題
Photo popup への利用
Occlusion, Object の利用により高精度化
まとめ
Occlusion/Boundary の算出– 一枚の画像から geometry, depth などを用いて算出– 高精度なセグメンテーション
Occlusion/Boundary の利用– セグメンテーションによるエラーの低減– 一般物体認識に有用
課題：– 群衆などから得られる Boundary の高精度化
top related