Raskar, Camera Culture, MIT Media Lab Camera Culture Ramesh Raskar Camera Culture Associate Professor, MIT Media Lab.

Raskar, Camera Culture, MIT Media Lab

Camera Culture

Ramesh Raskar

Camera CultureAssociate Professor, MIT Media Lab

Capturing and Sharing the Experience

• Computational Cinematography and Display– Future cameras for Movies and News– 6D displays

• Morphable Studios– Illumination and appearance on proxy geometry

• Programmable Movies– Enriching with Meta-data

• Performance Capture– ‘On-set’ natural motion capture, ‘Second Skin’

Ramesh Raskar, MIT Media Lab

Captured Blurred Photo

Refocused on Person

6D = light sensitive 4D display

One Pixel of a 6D Display = 4D Display

Desired Virtual Model

© Andrei State

Morphable Studios

ShaderLamps and BeingThere with UNC Chapel Hill

Towards ‘on-set’ performance capture

• 500 Hz with Id for each Marker Tag• Visually imperceptible tags + Natural lighting• Unlimited Number of Tags• Base station and tags only a few 10’s $

Traditional: High-speed IR Camera + Body markers

Second Skin: High-speed LED emitters+ Photosensing Body markers

Imperceptible Tags under clothing, tracked under ambient light

Capturing and Sharing the Experience

• Computational Cinematography and Display– Future cameras for Movies and News– 6D displays

• Morphable Studios– Illumination and appearance on proxy geometry

• Programmable Movies– Enriching with Meta-data

• Performance Capture– ‘On-set’ natural motion capture, ‘Second Skin’

Ramesh Raskar, MIT Media Lab

Mitsubishi Electric Research Laboratories Special Effects in the Real World Raskar 2006Inverse Optical Mo-Cap

DeviceHigh Speed Projector + Photosensing Markers

High Speed Camera + Reflecting/Emitting Markers

Params Location, Orientation, Illum Location

Settings

Natural SettingsAmbient Light Outdoors, Stage lighting

Imperceptible tags Hidden under wardrobe

Controlled LightingVisible, High contrast Markers

#of TagsUnlimited

Space LabelingUnique Id

LimitedNo Unique Id Marker swapping

SpeedVirtually unlimited Optical comm comps

LimitedSpecial high fps camera

CostLow Open-loop projectors Current: Projector/Tag=$100

High High bandwidth camera Current Camera: $10K

Traditional

Coded Aperture CameraCoded Aperture Camera

The aperture of a 100 mm lens is modified

Rest of the camera is unmodifiedInsert a coded mask with chosen binary pattern

Capturing and Sharing the Experience

• Morphable Studios– Illumination and appearance on proxy geometry– Separating content from physical proxy

• Programmable Movies– Cliplet aggregation– Storytelling cameras via meta-data– Making cameras and world intelligent– Long-distance bar-codes

• Performance Capture– ‘On-set’ motion capture, ‘Second Skin’– Natural environments– Lightweight technologies

• Computational Cinematography and Display– Future cameras for Movies and News– Universal software platform– 4D and 6D displays