Tridelity

1111 Tech Note Series

TRIDELITY Tech Note Series#2 –

ContentsContentsContentsContents

1 Introduction................................

1.1 Binocular Vision or Stereo Vision

1.2 Virtual Stereo ................................

1.3 Generating Stereo (3

1.4 Applications of Stereoscopy

1.4.1 Scientific Analysis and Visualization

1.4.2 Virtual Reality ................................

1.4.3 Computer-aided design

1.4.4 Digital Signage

2 Creating Stereoscopic Content

2.1 General Rule for rendering in Stereo

2.2 Camera ................................

2.3 Checklist for creating stereo content

3 Autostereoscopic Displays

3.1 How does it work? ................................

3.2 LCD Basics ................................

3.3 Multiplexing ................................

4 About TRIDELITY ................................

Tech Note Series - #2 Autostereoscopy Copyright 2011 TRIDELITY AG

Copyright 2011

TRIDELITY Tech Note Series– Autostereoscopy

................................................................................................

Binocular Vision or Stereo Vision ................................................................

................................................................................................

Generating Stereo (3-D) Content ................................................................

Applications of Stereoscopy ................................................................

Scientific Analysis and Visualization ................................

.............................................................................................

aided design ................................................................

Digital Signage ...........................................................................................

tereoscopic Content ................................................................

General Rule for rendering in Stereo ...............................................................

................................................................................................

ting stereo content ...............................................................

Autostereoscopic Displays ................................................................

...........................................................................................

................................................................................................

................................................................................................

................................................................................................

2013-05-03

Copyright 2011 TRIDELITY AG

TRIDELITY Tech Note Series

.............................................. 2

..................................... 2

.................................... 3

...................................... 3

............................................. 4

.......................................................... 4

............................. 4

............................................. 4

........................... 4

............................................... 6

............................... 6

.............................................. 7

............................... 9

.................................................... 12

........................... 12

...................................... 13

.................................... 14

.................................... 18

2222 Tech Note Series - #2 Autostereoscopy Copyright 2011 TRIDELITY AG

2013-05-03

1111 IntroductionIntroductionIntroductionIntroduction

In the recent past, 3-D technology has experienced a boom, especially in the consumer market. Early in 2010, the Korean display market research company Display-Bank added glasses-free 3-D solutions to their list of the most important display innovations for the next ten years. And it’s only a matter of time until companies in all industries adapt to the market changes.

1.11.11.11.1 Binocular VisionBinocular VisionBinocular VisionBinocular Vision or Stereo Visionor Stereo Visionor Stereo Visionor Stereo Vision

Usually the brain receives images from both eyes at the same time. These two

images are slightly different from each other due to eye separation (Binocular

Disparity). The brain uses these small differences to combine these views into one

and provide us with the notion of how far an object is (depth perception) and how fast

the object is getting closer or away from us (movement perception).

In order to test how important binocular vision is to us, simply close one eye and at

an arm length try bringing two pencil ends on top of each other. It is certainly more

difficult than when both eyes are open. This happens because your skill in judging

depth becomes poorer. By way of practice, these tasks may become easier and this

is only possible since there are many other clues that help us when judging depth,

the so called depth cues:

- Size of objectsSize of objectsSize of objectsSize of objects (We know how big objects are).

- ShadowShadowShadowShadow / Li/ Li/ Li/ Lightningghtningghtningghtning. Closer objects are brighter, distant ones dimmer. There a

number of other more subtle cues implied by lighting, the way a curved surface

reflects light suggests the rate of curvature, shadows are a form of occlusion.

- InterpositionInterpositionInterpositionInterposition (the partial blocking of a more distant object by a nearer object)

- Relative heightRelative heightRelative heightRelative height (things near the horizon give the impression they are distant)

- PerspectivePerspectivePerspectivePerspective. Objects get smaller the further away they are and parallel lines converge

in distance.

- DetailDetailDetailDetail. Close objects appear in more detail, distant objects less.

- Relative motionRelative motionRelative motionRelative motion. Objects further away seem to move more slowly than objects in the

foreground.

One thing that must be considered when creating stereo content is that not

everybody has binocular vision. There are cases where it might be reduced or even

completely lost, reasons include:

- Reduced or lost vision on one eye

- Loss of coordination of movement between both eyes

- Issues with the brain comparing images from both eyes


2013-05-03

Hence, when creating stereo content one has to make sure that binocular view alone

is not enough for a perfect stereoscopic view, the other depth cues should also be

used.

1.21.21.21.2 Virtual StereoVirtual StereoVirtual StereoVirtual Stereo

Using devices to generate stereostereostereostereo or binocular vision is a way to “trick” our brains in

order to create a sense of depth; however this view will never be as correct as in the

real world.

The problem with virtual stereo devices is that our eyes need to move in a way they

are not used to. Here is an example. Looking to an object close to the eyes; a person

will see it double until the eyes converge to it so that only one object is seen. This

process is called fusionfusionfusionfusion or convergenceor convergenceor convergenceor convergence.

The other task the eyes must do is to focus on an object so that it sharpens. This

process is called aaaaccommodationccommodationccommodationccommodation.

When using stereo displays, as expected, the eyes focus on one object but converge

to another. You might be asking yourself how this could happen but the answer is

quite simple. The eyes will try to accommodate to the display’s surface but the target

object is actually a projection that is displayed on this monitor and therefore they will

converge to the parallax of this object.

This might seem strange at first, but it’s important to mention it since some people

will feel a certain discomfort when using such a display for the first time.

1.31.31.31.3 Generating StereoGenerating StereoGenerating StereoGenerating Stereo ((((3333----DDDD)))) ContentContentContentContent

The process of supporting stereo can be divided basically into two steps: first is the

creation of one or more stereo pairs; in other words, the creation of the left and right

views; the second step is how to display these perspectives in order to reach the

desired stereo effect.

Fortunately, creating stereo content is no as complicated as you might think.

TRIDELITY offers cutting-edge software solutions, providing 3-D artists with all

necessary tools that solve the technical details, saving time which should be better

used for the creativity process.

We would like to avoid going too far, so we will not go into extensive details about

content creation now. But don’t worry; you will get more information on the next

chapters.


2013-05-03

1.41.41.41.4 Applications of StereoscopyApplications of StereoscopyApplications of StereoscopyApplications of Stereoscopy

1.4.11.4.11.4.11.4.1 Scientific Analysis and VisualizationScientific Analysis and VisualizationScientific Analysis and VisualizationScientific Analysis and Visualization

Scientific data is often complex and the relationship among co-related data can be

very difficult to visualize. Stereoscopy provides a great tool for presenting 3-D

information in a way that can be easily grasped. Data displayed in stereo provides

often new insights into the underlying process you are investigating. Besides,

stereoscopy provides a better way to communicate ideas with research colleagues.

1.4.21.4.21.4.21.4.2 Virtual RealityVirtual RealityVirtual RealityVirtual Reality

Computer stereoscopy provides a way to display objects as if they really physically

existed. A variety of systems can be profitably simulated.

The common example is the flight simulation. The system is usually composed by a

plane and a world that tries to simulate the characteristics of the real world without

the danger of having a learning pilot make mistakes which would be possibly end-up

badly.

1.4.31.4.31.4.31.4.3 ComputerComputerComputerComputer----aided designaided designaided designaided design

Nowadays, almost all design and construction disciplines use some kind of

stereoscopic method to facilitate their job. Through this exceptional viewing method,

the designer is able to scrutinize the current model and point out changes in its

shape which can be changed in real-time and rendered again.

To name just a few disciplines, stereoscopy can therefore help Architects design

better buildings making adjustments together with their customers; designers can

model more appealing cars while increasing efficiency through analysis and

simulations.

1.4.41.4.41.4.41.4.4 Digital SignageDigital SignageDigital SignageDigital Signage

In the past, the need for additional tools, such as shutter or red/green glasses, was

the main reason why 3-D technology did not make sense for the advertising industry.

However, TRIDELITY develops and manufactures displays that are able to present 3-

D content without any additional accessories. Thanks to the multi-view stereo

technology, the devices allow 3-D vision from multiple perspectives.

The SKOPOS Institute for market and communication research conducted a study on

advertising in 3-D in October 2010. In the study, 312 people were divided into two


2013-05-03

equal groups, each of whom were presented a commercial either in 3-D or in 2D.

The differences between the test groups were clear: the 3-D viewers found the

commercial to be modern, original and unique. After the test, 82% of the 3-D viewers

were convinced of the product. In the 2D group, only 64% were convinced. After the

3-D broadcast, viewers also felt more of a desire to try the advertised product; in

other words, the purchase probability was significantly greater.

In addition, in both test groups, 43% said they would also like to watch 3-D at home.

These test results show that there is generally a strong willingness to view 3-D

content, which can thus be exploited very advantageously for the advertising

industry.


2013-05-03

2222 Creating Stereoscopic ContentCreating Stereoscopic ContentCreating Stereoscopic ContentCreating Stereoscopic Content

A 3-D film immerses you in the scene, with a greatly enhanced sense of physical presence and participation. When most people think of 3-D films, they think first of the gimmick shots -- objects or characters flying, floating or poking out into the audience. In fact, in a good stereo movie, these shots should be the exception rather than the rule. Watching a stereo movie is looking into an alternate reality through a window. – James Cameron Apr. 2008

2.12.12.12.1 General Rule for rendering in StereoGeneral Rule for rendering in StereoGeneral Rule for rendering in StereoGeneral Rule for rendering in Stereo

Rendering ones first stereo scene will be certainly a try and error affair. The

following approach might help avoiding some common mistakes. First step is to

choose the camera aperture, values between 45 and 60 degrees should enough. At

this point you should choose the focal plane; this is the plane where the objects will

appear to be 2D (zero parallax). Objects in front of this plane will appear floating in

front of the screen (negative parallax); objects behind the plane will appear behind

the screen (positive parallax).

The following picture illustrates this principle.

How close objects can come to the camera depends somewhat on the display

system.

Screen

(Projection Plane)

Positive Parallax

Negative Parallax

Camera 1

Camera 2


2013-05-03

A common measure of the stereo effect is the parallax angle. The parallax angle is

for example used by Astronomers in order to find the distance from the Earth to a

certain star.

For easy convergence for the majority of people, the absolute value of this angle

should not exceed 0.40 degrees for a 5-view autostereoscopic display. Note that this

angle is positive for positive parallax and negative for negative parallax. For negative

parallax, we recommend a maximum of –0.25 degrees in order to avoid cross-talk.

2.22.22.22.2 CameraCameraCameraCamera

Vector arithmetic will help us bring together various cases that arise when creating

stereo projections into a single expression. If you are not familiar with it, we

recommend having a look back at the basics before continuing.

A camera could be defined by its projection axis, its view volume and focus. Using a

very simplistic model, we could define a camera by drawing a pyramid as the viewing

volume that is sliced by the near and far planes like the picture below.

What you can notice straight away is that this perspective is symmetric and it’s

perfectly suited for non-stereo projections. Note however that although the

projection axis and the view frustum are both needed in order to generate the final

perspective, they are not dependent in each other and might be configured

separately.

In order to generate stereo views we will need this feature in such a way that the

perspective axis will not be changed. On the other hand, the view frustum will be


2013-05-03

redefined so that different parts of the scene will be displayed through a so-called

asymmetric frustum as you can see in the picture below.

The interesting aspect of this kind of projection is that the depth order, position and

shape of the objects will be the same in any singular view frustum as long as the

projection axis stays the same. This characteristic is very important in order to

generate correct stereo views.

However, many software applications do not support such kind of asymmetric

frustum. In these cases, we need to use symmetric frustum and extend the

horizontal field of view for each eye. After rendering, those parts of the image must

be trimmed off.

In this case, let’s call the camera aperture β, the eye separation s, the intended

width w and the focal distance f, the part to be trimmed delta is:

�� = �2�tan(�2)

For example, if w = 960, f = 30, s = f/120, β = 60 degrees, then delta is 7 pixels and the

new camera aperture 60.36115.

When using software that do support asymmetric frustum like OpenGL, the

transformation can be represented by the following calculations.

Half of the width of the projection plane can be represented by:

ℎ_��ℎ = �� ∗ tan(��2 )


2013-05-03

In this case, delta can be represented by:

�� = �� ∗ �� In OpenGL for example, we could use the command glFrustrum(left, right, bottom,

top, N, F) in order to create a viewing frustum. The projection matrix looks like this:

!!!!"

2#��$ℎ� − �� 0

0 2#�� − '��(

��$ℎ� + ��$ℎ� − �� 0�� + '��(�� − '��( 0

0 00 0−(* + #)* − # −2*#* − #−1 0 ,

----.

In order to get the right eye projection, supposing an aspect ratio AR, we could

simply define left and right being:

/0 = 12 ∗ ℎ34567 − 0.5(��) Where left = -CO and right = CO. Now we simply move the camera using half the

camera separation:

��(��. :; = ��(��. : + ��2

If this seems a bit confusing at first, don’t worry. TRIDELITY offers a complete

OpenGL example in C++ including the source codes.

If you are a content creator, the plug-ins will do the complete calculation for you and

this information will just help you understand what’s going on under the hood.

2.32.32.32.3 Checklist for creating stereo contentChecklist for creating stereo contentChecklist for creating stereo contentChecklist for creating stereo content

Following we will present you some guidelines that must be following in order to

create stereo content that are easy to the eyes.

1. Cross-talk

Stereoscopy is never perfect; there will be always some kind of leakage

from one perspective view reaching the wrong eye. Make sure to limit the

parallax before cross-talking gets too high.


2013-05-03

2. Slow it down!

Make sure that things that have high parallax are not moving too fast.

Especially if they have negative parallax.

3. Interposition

This is one of the most common errors nowadays. Content creators tend

to add objects to stereo scenes in post-production and they forget the

objects that are in front of others should hide them. How often have I

seen objects going through other objects and completely destroying a

good 3-D movie.

4. Interference

Sometimes there might be structures that repeat themselves in a certain

frequency, like the bars on a jail. A problem might occur if this frequency

matches the parallax separation. This can lead to very disturbing images

and increase the possibility of people getting headaches.

5. Camera changes

Event changing the camera settings too often. You might already have

noticed that movie trailers in 3-D are usually more difficult to see than

the movie itself. This is due to the frequent cut to scene with different

focus and parallax values.

6. Negative Parallax

Avoid using too much of negative parallax on longer movies and make

sure these objects are placed in the center of the screen. Viewers will

actually find positive parallax (behind the screen) more comfortable.

7. Contrast

Avoid high contrast on scenes with high parallax. One of the main

problems here is due to the LCD technology. Neighbor pixel will tend to

light up a bit causing ghosting which will destroy the parallax.

8. A 3-D Display is a window!

Treat the 3-D display as a window. Things that should come out through

the windows must fit completely inside it. If you try to move something

out of the window and it touches the border, the object should stay where

it is!


2013-05-03

9. Maximum Parallax

Try not to exceed the 1% of the horizontal active area (resolution) for

positive parallax and 1.5% of positive parallax on a display screen.

10. Toe-In

Never use toe-in cameras. This will result in bad 3-D and cause eye-

strain due to vertical parallax. Make sure you always use parallel

cameras.


2013-05-03

3333 Autostereoscopic DisplaysAutostereoscopic DisplaysAutostereoscopic DisplaysAutostereoscopic Displays

“There's something that 3D gives to the picture that takes you into another land and you stay there and it's a good place to be...” - Martin Scorcese 04/2012

3.13.13.13.1 How does it work?How does it work?How does it work?How does it work?

The content is produced by taking “pictures” from five slightly different positions,

making them look slightly different from each other. Each of these pictures is then

shown at the same time on a modified LCD display, and the final output is compiled

using a special multiplex pattern where only some information from each picture is

selected.

A specially developed optical barrier element is then placed on top of the LCD

display and is responsible for splitting the pictures so that different images are

projected onto each eye of the viewer. Due to the fact that your left eye sees a

different picture than your right eye, and therefore has another perspective, the

brain is capable of merging both perspectives into a 3-D scene.


2013-05-03

Benefits:

• The parallax barrier replaces the cumbersome eye-ware required

nowadays.

• Displaying 5 different views allows many users to see 3-D comfortably

while keeping an acceptable resolution in opposite to 8 view systems,

• The monitor does not distort content in opposite to lenses-based systems.

• In contrary to most lenses based displays, 2-D content works seamlessly.

3.23.23.23.2 LCD BasicsLCD BasicsLCD BasicsLCD Basics

The main technology behind all autostereoscopic displays that exist today is based

on multiplexing pixels or sub-pixels. For example, everybody knows the old 3-D

videos in Anaglyph (red and green glasses). These videos can also be represented by

a kind of multiplex schema. Before we start giving examples, let’s check how a LCD

based Monitor works.

The liquid crystals used by LCDs are able to change their brightness depending on

the electrical voltage applied to them. These crystals are equipped with a special

transistor (TFT) which makes it possible to change their direction of the liquid and

therefore changing the transparency of each liquid-crystal based pixel. The pixels on

a LCD are arranged as a matrix, as shown in the following picture.

LCDs are based on a variant of the additive color model. This model basically uses

red, green and blue light in order to produce the other colors. On LCDs however, the

colors are not mixed. However, due to the subpixes beeing so small, we can assume

this model. For the purpose of this document, therefore we will call this model from

now on the “RGB color model”.


2013-05-03

The additive model is somehow aukward for most people who worked with inks

already. When using inks, when you add green, blue and red you end up with some

kind of gray, on the RGB model however, you will end up with white. Well, you may

ask yourself, how you produce black. This is very simple, you simply turn off all color

components.

3.33.33.33.3 MultiplexingMultiplexingMultiplexingMultiplexing

As mentioned in section 3.2, most autostereoscopic displays are based on some kind

of multiplexing. In order to give you an example, let’s take the well-known anaglyph

3-D technology used still today in many ways, as for example on YouTube.

According to Wikipedia: “Anaglyph images are used to provide a stereoscopic 3-D

effect, when viewed with glasses where the two lenses are different (usually

chromatically opposite) colors, such as red and cyan”

Well, if you remember how a LCD is constructed, it is very easy for us now to create

a filter that will change the left and right pictures so that we “hide” the left picture

using red, and the right picture using cyan. Let’s do it by multiplying the left picture

by a vector, supposing a RGB pixel is represented by [R, G, B]:

For each pixel:

Anaglyph_Left[i] = Color_Left[i] * Filter_Left

Anaglyph_Right[i] = Color_Right[i] * Filter_Right


2013-05-03

End

FinalPicture = Anaglyph_Left + Anaglyph_Right

Where:

Filter_Left = [1, 0, 0]

Filter_Right = [0, 1, 1]

The example above simply erases all blue and green color information of the left

image and the red from the right image and then adds both of them up creating a

new image which contains two perspectives. This is done by using two masks, one

for the right [0, 1, 1] and one for the left [1, 0, 1].

Autostereoscopic displays do more or less the same, but in a way that no glasses are

needed. The mask is designed in such a way that it fits the parallax barrier which is

placed on top of the display.

On single-viewer displays the mask might be as simple as just removing zeroing all

pixels on even columns for the left image and all pixels on the even column for the

right image. On multi-viewer it gets a bit more complicated.


2013-05-03

Let’s suppose we have a 5 view autostereoscopic display whose Mask is a 5x5 Matrix.

This means, that we have one Matrix for each perspective, one for the first image,

one for the second and so one. The pseudo-code for this multiplexing will be

something like the following:

For each perspective

For each pixel

m_perpective[i] = perspective[n] * matrix[n]

End

End

The good thing about it is that we can do the operation very fast using parallel

computing like CUDA or OpenCL.

Tridelity offers an example of using these techniques. If you are interested, please

get in contact with us to get the complete example using C++ and OpenGL

(Pixelshader). Following, is the source code of a typical pixel shader for multiplexing

views:

uniform sampler2D tex0;

uniform sampler2D tex1;

uniform vec2 texsize0;



void main()

{

float y9 = texsize0.y;

float x9 = texsize0.x;

vec2 coord0 = vec2(gl_TexCoord[0].x, gl_TexCoord[0].y + y9 + y9);

vec2 coord1 = vec2(gl_TexCoord[0].x + x9, gl_TexCoord[0].y + y9);

vec2 coord2 = vec2(gl_TexCoord[0].x, gl_TexCoord[0].y + y9);

vec2 coord3 = vec2(gl_TexCoord[0].x + x9, gl_TexCoord[0].y);

vec2 coord4 = vec2(gl_TexCoord[0].x, gl_TexCoord[0].y);

vec3 color1 = texture2D(tex0, coord0).rgb;



2013-05-03




float xx = mod(gl_FragCoord.x, texsize2.x) * texsize1.x;

float yy = mod(gl_FragCoord.y, texsize2.y) * texsize1.y;

vec4 lookup1 = texture2D(tex1, vec2(xx,yy));

vec4 lookup2 = texture2D(tex1, vec2(xx + (texsize2.x * texsize1.x),yy));

vec4 lookup3 = texture2D(tex1, vec2(xx + 2.0 * (texsize2.x * texsize1.x),yy));



gl_FragColor.rgb = color5 * lookup5.rgb + color4 *lookup4.rgb + color3 *lookup3.rgb

+ color2 *lookup2.rgb + color1 *lookup1.rgb;

}";


2013-05-03

4444 About TRIDELITYAbout TRIDELITYAbout TRIDELITYAbout TRIDELITY

"Who the hell wants to hear actors talk?" -- H. M. Warner, Warner Brothers, 1927

TRIDELITY AG is a provider of glasses-free auto-stereoscopic 3-D screens; the

company has been pioneering the development of this cutting-edge technology for

years, offering the highest quality 3-D solutions. In addition to related technologies

and services, TRIDELITY also provides an extensive hardware range of solutions for

single-viewer and multi-viewer purposes, with unprecedented viewing comfort,

exceptional 3-D depth and the highest resolution in the industry. For more

information, visit www.tridelity.com.

Tridelity

Technology

virtual stereo

stereo vision1

tridelity agcopyright

binocular vision

d content

general rule

scientific analysis

virtual reality