Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light.

Johnny Lee, Scott Hudson, Paul DietzCarnegie Mellon University

Mitsubishi Electric Research LabsUIST 2007 – Newport, RI

Hybrid Infrared and Visible Light Projection for Location Tracking

Hybrid Projection

infrared visibleone projector

But, why?

Projector-Based Location Discovery [Lee, UIST’04]

Light sensors

Electronics & computer

Projector

Projector-Based Location Discovery

calibration free: - no computer vision - no alignment - no manual input

Scalable and robust

Moveable Surfaces

[Lee, UIST 2005]

- calibration free- no external tracker- interactive content

Drawbacks

Location Discovery [‘04] Moveable Surfaces [‘05]

Caustic B&W patternsMomentary movement

Incremental tracking

Ideally

Full-screenLocation Patterns

Full-screenApplication Content

Hybrid Projection

infrared visible

for the computer for the human

one projector

Infrared & Visible Projection

Infrared Visible

9

Infrared & Visible Projection

Infrared Visible

10

Infrared & Visible Projection

Infrared Visible

11

Infrared & Visible Projection

Infrared Visible

12

Infrared & Visible Projection

Infrared Visible

13

Infrared & Visible Projection

Infrared Visible

14

Infrared & Visible Projection

Infrared Visible

15

Infrared & Visible Projection

Infrared Visible

16

Infrared & Visible Projection

Infrared Visible

17

Infrared & Visible Projection

Infrared Visible

18

How?

Light Source: Lamps

Xenon Arc Lamp

Light Technologies

More Efficient and Better Lifespan

Products emerging on market this year, 2007

IR and Visible Light LEDs 1000 LEDs/mm2

University of Strathclyde, Institute of Photonics

DMD LED ArrayLens

Projection optics

Our Dev Kit: 180 binary images/sHigh-Speed Dev Kit: 16,500 binary images/sProduction Unit: +50,000 binary images/s

1024x768 area = 20 binary images60Hz tracking = 2.4% duty cycle of production DMDRequired changes to commercial designs would be minimal.

Demo of CapabilityUses a second projector for visible content

Inherent Multi-Stylus Tracking

Non-Planar and Discontinuous Surfaces

Static IR Patterns

Concept ApplicationsSimulated using external tracking (calibration)

[Siggraph 2004]

Hand-held projection with photosensitive tags

Foldable Interactive Displays [submitted to CHI]

Acknowledgements

Johnny Chung Lee

[email protected]

Funded in part by the National Science Foundation under grants IIS-0121560 and IIS-0325351

Funded in part by Mitsubishi Electric Research Labs

Other ways to make invisible patterns

Other non-visible wavelengthsSteganography

– Color shifting– Noise encoding

Bit Timing– Synchronization may be difficult

Camera-Based TrackingRequires calibrationRequires markers for segmentation

– IR sensors + transmitter is less power than 4 IR LEDs

Does not provide IDLimitation on the number of pointsLimitation on tracking rateLimitation on scene/target complexityResolution is not as scalableLess optically robust

– Optical path geometry and variable illumination

Projector vs Camera Tracking

• Sensors provide point ID• Independent of scene/surface complexity

Space-Labeling Projectors

11 Infrared LED slide projectors

Potentially Low-cost

Per axis: 500Hz tracking at 10-bits

Outdoor motion tracking

37

Binary Gray

0000

0001

0010

0011

0100

0101

0110

0111

1000

1001

1010

1011

1100

1101

1110

1111

0000

0001

0011

0010

0110

0111

0101

0100

1100

1101

1111

1110

1010

1011

1001

1000

Binary Gray

0000

0001

0010

0011

0100

0101

0110

0111

1000

1001

1010

1011

1100

1101

1110

1111

0000

0001

0011

0010

0110

0111

0101

0100

1100

1101

1111

1110

1010

1011

1001

1000

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Binary Gray

Pixart Chip

1024x768 resolution100Hz tracking4 points

Only 4 pointsHigh power LED points