3D Vision. 3D Perception: Illusions Block & Yuker.

3D Vision3D Vision

3D Perception: Illusions3D Perception: Illusions

Block & YukerBlock & Yuker



















3D Perception: Conclusions3D Perception: Conclusions

• Perspective is assumedPerspective is assumed

• Relative depth orderingRelative depth ordering

• Occlusion is importantOcclusion is important

• Local consistencyLocal consistency

3D Perception: Stereo3D Perception: Stereo

• Experiments show that absolute depth Experiments show that absolute depth estimation not very accurateestimation not very accurate– Low “relief” judged to be deeper than it isLow “relief” judged to be deeper than it is

• Relative depth estimation very accurateRelative depth estimation very accurate– Can judge which object is closer for stereo Can judge which object is closer for stereo

disparities of a few seconds of arcdisparities of a few seconds of arc

3D Computer Vision3D Computer Vision

• Accurate (or not) shape reconstructionAccurate (or not) shape reconstruction

• SegmentationSegmentation

• Easier (?) to match in 3D than in 2DEasier (?) to match in 3D than in 2D– OcclusionOcclusion

– ShadingShading

– Smaller database of examplesSmaller database of examples

3D Data Types3D Data Types

• Point DataPoint Data

• Volumetric DataVolumetric Data

• Surface DataSurface Data

3D Data Types: Point Data3D Data Types: Point Data

• ““Point clouds”Point clouds”

• Advantage: simplest data typeAdvantage: simplest data type

• Disadvantage: no information onDisadvantage: no information onadjacency / connectivityadjacency / connectivity

3D Data Types: Volumetric Data3D Data Types: Volumetric Data

• Regularly-spaced grid in (x,y,z): “voxels”Regularly-spaced grid in (x,y,z): “voxels”

• For each grid cell, storeFor each grid cell, store– Occupancy (binary: occupied / empty)Occupancy (binary: occupied / empty)

– DensityDensity

– Other propertiesOther properties

• Popular in medical imagingPopular in medical imaging– CAT scansCAT scans

– MRIMRI

3D Data Types: Volumetric Data3D Data Types: Volumetric Data

• Advantages:Advantages:– Can “see inside” an objectCan “see inside” an object

– Uniform sampling: simpler algorithmsUniform sampling: simpler algorithms

• Disadvantages:Disadvantages:– Lots of dataLots of data

– Wastes space if only storing a surfaceWastes space if only storing a surface

– Most “vision” sensors / algorithms returnMost “vision” sensors / algorithms returnpoint or surface datapoint or surface data

3D Data Types: Surface Data3D Data Types: Surface Data

• PolyhedralPolyhedral– Piecewise planarPiecewise planar

– Polygons connected togetherPolygons connected together

– Most popular: “triangle meshes”Most popular: “triangle meshes”

• SmoothSmooth– Higher-order (quadratic, cubic, etc.) curvesHigher-order (quadratic, cubic, etc.) curves

– Bézier patches, splines, NURBS, subdivision Bézier patches, splines, NURBS, subdivision surfaces, etc.surfaces, etc.


• Advantages:Advantages:– Usually corresponds to what we seeUsually corresponds to what we see

– Usually returned by vision sensors / Usually returned by vision sensors / algorithmsalgorithms

• Disadvantages:Disadvantages:– How to find “surface” for translucent How to find “surface” for translucent

objects?objects?

– Parameterization often non-uniformParameterization often non-uniform

– Non-topology-preserving algorithms difficultNon-topology-preserving algorithms difficult


• Implicit surfaces (cf. parametric)Implicit surfaces (cf. parametric)– Zero set of a 3D functionZero set of a 3D function

– Usually regularly sampled (voxel grid)Usually regularly sampled (voxel grid)

• Advantage: easy to write algorithms Advantage: easy to write algorithms that change topologythat change topology

• Disadvantage: wasted space, timeDisadvantage: wasted space, time

2½-D Data2½-D Data

• Image: stores an intensity / color alongImage: stores an intensity / color alongeach of a set of regularly-spaced rays in each of a set of regularly-spaced rays in spacespace

• Range image: stores a Range image: stores a depthdepth along alongeach of a set of regularly-spaced rays in each of a set of regularly-spaced rays in spacespace

• Not a complete 3D description: does notNot a complete 3D description: does notstore objects occluded (from some viewpoint)store objects occluded (from some viewpoint)

• View-dependent scene descriptionView-dependent scene description


• This is what most sensors / algorithmsThis is what most sensors / algorithmsreallyreally return return

• AdvantagesAdvantages– Uniform parameterizationUniform parameterization

– Adjacency / connectivity informationAdjacency / connectivity information

• DisadvantagesDisadvantages– Does not represent entire objectDoes not represent entire object

– View dependentView dependent


• Range imagesRange images

• Range surfacesRange surfaces

• Depth imagesDepth images

• Depth mapsDepth maps

• Height fieldsHeight fields

• 2½-D images2½-D images

• Surface profilesSurface profiles

• xyzxyz maps maps

• ……

Range Acquisition TaxonomyRange Acquisition Taxonomy

RangeRangeacquisitionacquisition

ContactContact

TransmissiveTransmissive

ReflectiveReflectiveNon-opticalNon-optical

OpticalOptical

Industrial CTIndustrial CT

Mechanical Mechanical (CMM, jointed arm)(CMM, jointed arm)

RadarRadar

SonarSonar

UltrasoundUltrasound

MRIMRI

Ultrasonic trackersUltrasonic trackersMagnetic trackersMagnetic trackers

Inertial Inertial (gyroscope, accelerometer)(gyroscope, accelerometer)

Range Acquisition TaxonomyRange Acquisition Taxonomy

OpticalOpticalmethodsmethods

PassivePassive

ActiveActive

Shape from X:Shape from X:stereostereomotionmotionshadingshadingtexturetexturefocusfocusdefocusdefocus

Active variants of passive methodsActive variants of passive methodsStereo w. projected textureStereo w. projected textureActive depth from defocusActive depth from defocusPhotometric stereoPhotometric stereo

Time of flightTime of flight

TriangulationTriangulation

Optical Range Acquisition Optical Range Acquisition MethodsMethods

• Advantages:Advantages:– Non-contactNon-contact

– SafeSafe

– Usually inexpensiveUsually inexpensive

– Usually fastUsually fast

• Disadvantages:Disadvantages:– Sensitive to transparencySensitive to transparency

– Confused by specularity and interreflectionConfused by specularity and interreflection

– Texture (helps some methods, hurts others)Texture (helps some methods, hurts others)

StereoStereo

• Find feature in one image, search along Find feature in one image, search along epipole in other image for epipole in other image for correspondencecorrespondence

StereoStereo

• Advantages:Advantages:– PassivePassive

– Cheap hardware (2 cameras)Cheap hardware (2 cameras)

– Easy to accommodate motionEasy to accommodate motion

– Intuitive analogue to human visionIntuitive analogue to human vision

• Disadvantages:Disadvantages:– Only acquire good data at “features”Only acquire good data at “features”

– Sparse, relatively noisy data (correspondence is hard)Sparse, relatively noisy data (correspondence is hard)

– Bad around silhouettesBad around silhouettes

– Confused by non-diffuse surfacesConfused by non-diffuse surfaces

• Variant: multibaseline stereo to reduce Variant: multibaseline stereo to reduce ambiguityambiguity

Shape from MotionShape from Motion

• ““Limiting case” of multibaseline stereoLimiting case” of multibaseline stereo

• Track a feature in a video sequenceTrack a feature in a video sequence

• For For nn frames and frames and ff features, have features, have22nnff knowns, 6 knowns, 6nn+3+3ff unknowns unknowns

Shape from MotionShape from Motion

• Advantages:Advantages:– Feature tracking easier than Feature tracking easier than

correspondence in far-away viewscorrespondence in far-away views

– Mathematically more stable (large baseline)Mathematically more stable (large baseline)

• Disadvantages:Disadvantages:– Does not accommodate object motionDoes not accommodate object motion

– Still problems in areas of low texture, in Still problems in areas of low texture, in non-diffuse regions, and around silhouettesnon-diffuse regions, and around silhouettes

Shape from ShadingShape from Shading

• Given: image of surface with known, Given: image of surface with known, constant reflectance under known point constant reflectance under known point lightlight

• Estimate normals, integrate to find Estimate normals, integrate to find surfacesurface

• Problem: ambiguityProblem: ambiguity

Shape from ShadingShape from Shading

• Advantages:Advantages:– Single imageSingle image

– No correspondencesNo correspondences

– Analogue in human visionAnalogue in human vision

• Disadvantages:Disadvantages:– Mathematically unstableMathematically unstable

– Can’t have textureCan’t have texture

• ““Photometric stereo” (active method) Photometric stereo” (active method) more practical than passive versionmore practical than passive version

Shape from TextureShape from Texture

• Mathematically similar to shape from shading, Mathematically similar to shape from shading, but uses stretch and shrink of a (regular) but uses stretch and shrink of a (regular) texturetexture

Shape from TextureShape from Texture

• Analogue to human visionAnalogue to human vision

• Same disadvantages as shape from Same disadvantages as shape from shadingshading

Shape from Focus and DefocusShape from Focus and Defocus

• Shape from focus: at which focus Shape from focus: at which focus setting is a given image region setting is a given image region sharpest?sharpest?

• Shape from defocus: how out-of-focus is Shape from defocus: how out-of-focus is each image region?each image region?

• Passive versions rarely usedPassive versions rarely used

• Active depth from defocus can beActive depth from defocus can bemade practicalmade practical

Active Optical MethodsActive Optical Methods

• Advantages:Advantages:– Usually can get dense dataUsually can get dense data

– Usually much more robust and accurate Usually much more robust and accurate than passive techniquesthan passive techniques

• Disadvantages:Disadvantages:– Introduces light into scene (distracting, etc.)Introduces light into scene (distracting, etc.)

– Not motivated by human visionNot motivated by human vision

TerminologyTerminology

• Range acquisition, shape acquisition, Range acquisition, shape acquisition, rangefinding, range scanning, 3D rangefinding, range scanning, 3D scanningscanning

• Alignment, registrationAlignment, registration

• Surface reconstruction, 3D scan Surface reconstruction, 3D scan merging, scan integration, surface merging, scan integration, surface extractionextraction

• 3D model acquisition3D model acquisition

Related FieldsRelated Fields

• Computer VisionComputer Vision– Passive range sensingPassive range sensing

– Rarely construct complete, accurate modelsRarely construct complete, accurate models

– Application: recognitionApplication: recognition

• MetrologyMetrology– Main goal: absolute accuracyMain goal: absolute accuracy

– High precision, provable errors more High precision, provable errors more important than scanning speed, complete important than scanning speed, complete coveragecoverage

– Applications: industrial inspection, quality Applications: industrial inspection, quality control, as-built modelscontrol, as-built models

Related FieldsRelated Fields

• Computer GraphicsComputer Graphics– Often want complete modelOften want complete model

– Low noise, geometrically consistent model Low noise, geometrically consistent model more important than absolute accuracymore important than absolute accuracy

– Application: animated CG charactersApplication: animated CG characters

3D Vision. 3D Perception: Illusions Block & Yuker.

Documents