Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL.

Image formation& Geometrical Transforms

Francisco Gómez JMMS

U. Central y UJTL

Digital images

• Arrays of numbers

• Numbers can represent:– Intensity (gray level)– Range– X-ray absortion

coefficient– Amount of light

Human eye

www.youreyescenter.com

Color sensors

Intensitysensors

100,0000,000 100,0000,000

Distribution of cones and rods

Sensor simplified

Pupil Limit amount of light and incidence angle

Lens Focus the light to a single image point

Fotosensitive surface Film

Digital cameras – sensor type

Sensor type

CCD type (Charge coupled device)Charge is accumulated during exposure

CMOS type (complementary metal oxide on silicon)Light affects the conductivity (or gain) of each photodetector

www.parentesis.com

Pixel!

To take into account: Number and size of sensor elements – Chip size – ADC resolution

Pinhole camera

The pinhole camera and camera obscura principle illustrated in 1925, in The Boy Scientist.

Pinhole camera

The «Camera obscura» was used by renaissance paienters to help to understandPerspective projection

Pinhole and lens model

https://foundationsofvision.stanford.edu/appendix/

Pinhole

Given a projection plane parallel to X-Y located at distance f where the point (p1,p2,p3) is going to be projected, i.e, which is the value for (u,v)?

u=(p1/p3)f and v=(p2/p3)f

Pinhole and lens model

This is the CCD sensor units in mmImage plane coordinate

This is the real object in m

Example

• Camera– Focal length: 5mm

• You have an scene point located at (1m,2m,5m)– Where at the image plane coordinate this point is going to be

located?– If the image plane is 10mm,10mm which is the FoV?– A building is 100m wide. How far away do we have to be in order

that it fills the field of view?

(0,0)

Optical center (cx,cy)Optical center

Image buffer

Optical center

digital image

Pixel

Image Origin

Image Buffer

Example

• A camera observes a rectangle 2m away– The rectangle is known to be 50 cm x 30 cm– If the image in the rectangle measures 60 x 15

pixels• Where is located the focal length in pixels?

Other kind of sensors (Color camera)

www.eoshd.com

Other kind of sensors (Kinect)

http://123kinect.com/kinect-forums/thread-569.html

Other kind of sensors

http://mesh.brown.edu/3dpgp-2009/homework/hw2/figures/teaser.jpg

Other kind of sensors

http://images.anandtech.com/doci/4057/StructuredLightSystem.png

Other kind of sensors (MRI)

http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html

Other kind of sensors (MRI)

http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html

Other kind of sensors (MRI)

https://electrosome.com/light-field-camera/

Other kind of sensors

All is about tranform between frames

2D to 2d Transforms (Rigid)

(1,1) (?,?)

2

2

Dx

(x’,y’)(?,?)

Dy q

Preserves shape and sizeNumber of degrees of freedom (3): t=(Dx,Dy) and q

2D to 2d Transforms (Euclidean - Rigid)

q

R is orthonormalTranspose is the inverseHow to invert the transform?

Homogeneous coordinates

- Homogeneous coordinates simply add an extra element 1- If during operations the third element is different of 1 divide by this number- This representation is quite convenient to represent transformations

to

Homogeneous coordinates

Example

• Transform the image point (10,40) using a rotation of 90 degrees and a translation of (15,-60)

2D to 2d Transforms (Similarity - Scaled)

Preserve angles but not distances

2D to 2d Transforms (Afinne)

Models rotation, translation, scaling, shearing, and reflection

Example

I = imread('cameraman.tif'); tform = maketform('affine',[1 0 0; .25 1.5 0; 0 0 1]); J = imtransform(I,tform); imshow(I), figure, imshow(J)

2D to 2d Transforms (Afinne)

Models rotation, translation, scaling, shearing, and reflection

2D to 2d Transforms (Projective - Homography)

What preserves?

3d to 3d Transforms

• Coordinate frames– {A}, {B}

• How to describe points in {B} respect to {A}– We need a t– Rotation matrix R

3d to 3d Transforms (Rotation)

http://upload.wikimedia.org/wikipedia/commons/thumb/c/c1/Yaw_Axis_Corrected.svg/2000px-Yaw_Axis_Corrected.svg.png

3 degrees of freedom! But we need a convention

Rotations around the axis

Rotation around the axis

• A rotation matrix can be expressed using a 3 x3 matrix

Rotation in XRotation in YRotation in Z

Rotational transformation

• R represents a rotational transformation of frame A to frame B

From

From

Point represented in frame B

Point represented in frame A

Transforming a 3d point

In homogeneous coordinates

Homogeneous

What is the inverse?

Example

• Let p=(2,2,5) in frame A• Frame B is located at (10,-5,6) and rotated 65

degrees around z-axis with respect to frame A• Where is point p located respect to frame B?

Transformations are maps

A B

Transformation are answers to the question:If there are points in A how they can be written in B?

Small angle approximation

sin( )= , q q sin( )=1q

Useful if you want to rotate objects in a video

Recovering angles

3d to 2d Transforms

Projects 3d points into 2d

Perspective projection – Intrinsic matrix

Optical center (cx,cy)

How can we write the perspective projection as a transform? orHow to project 3D points represented in the coordinate system attached to the camera, to the 2D image plane?

Extrinsic cameraIf 3D points are in world coordinates, we first need to transform them to camera coordinates

We can write this as an extrinsic camera matrix, that does the rotation and translation, then a projection from 3D to 2D

World to camera

What about color?

Images in matlab

RGB color space

RGB color space

Hue

Purity

Luminosity