Factors Influencing Virtual Reality Immersion

Factors Influencing Virtual Reality Immersion 2013

1

F A C T O R S I N F L U E N C I N G V I R T U A L R E A L I T Y I M M E R S I O N

John Guillermo Lara Rojas

Fontys University of Applied Sciences, Eindhoven, The Netherlands.

ABSTRACT

Virtual Reality (VR) immersion is critical when evaluating a VR system. To be able to find and analyze immersion factors, we firstly need to understand how humans perceive the world that surrounds us when a sensori stimuli is present and then compare to what

extent we can produce the same perception in a person while using a virtual environment (VE), but without confusing the subjective concept of presence. In

consequence raising the question why we cannot see the virtual world as a real one, like ours? Although the high immersion VR systems such as CAVE are present nowadays. Here I present a discussion–analysis taking Pausch, Proffitt and Williams experiment

where they quantify immersion concluding with strong points. Two other active researchers Bowman and McMahan present their interesting findings to take advantage of the benefits of immersion clarifying this paper’s objective and helping me to conclude

comfortably about the topic.

------------------------------------------------------------------------------------------------------------------

INTRODUCTION

When the National Academy of Sciences (NAS), a private non-profit society of distinguished scholars engaged in scientific and engineering research, recommended an agenda to determine when VR systems are better than desktop displays and stated that without scientific grounding many millions of dollars could be wasted, the commitment of 3 researchers namely Randy Pausch, Dennis Proffitt, George Williams from University of Virginia submitted a research paper “Quantifying Immersion in Virtual Reality” in 1997 where they took a step towards quantifying “immersion” or the sense of "being there" although this is a concept inaccuracy confusing Immersion with Presence according with professor of Virtual Reality Mel Slatter.

The experiment they conducted had basically the goal to search for a target in heavily camouflaged scenes using a VR system (HMD) and a desktop display. In any given search, there was a 50/50 chance that the target was somewhere in the scene. The user's job was to either find the target or claim no target was present. The results were outstanding when they point out a strong conclusion among others: “Users who practiced first with the traditional display negatively transferred that experience and


2

performed worse when using VR. This negative transfer may be relevant in applications that use desktop 3D graphics to train users for real-world tasks”. A clear example is airlines using VR with poor immersive devices and systems (a poor immersive device or system does not implicitly means they are cheap ones, just less expensive than full immersive systems) to train their pilots.

But hardware (input devices) alone does not totally comprise immersion. We need humans to control these virtual environments in a natural way like in reality through user interfaces but this is not an easy issue to develop although quite a lot of research has been made so far. I will also consider user interfaces as a factor of immersion in this paper.

So the degree of immersion it’s a really important issue in virtual reality systems but how can we determine the level of immersion a VR system has, or the feeling of “being there”? Pausch, Proffitt and Williams give us, in their research paper, a good idea how to approach to this question by measuring time response on each situation.

However, we need to clearly understand what immersion involves in VR systems. Therefore this research paper is intended to find out factors that influence VR immersion and then give an in-depth analysis about those factors to investigate why we still can perceive virtual environments as fake ones despite the advance research made in the technological field and especially in human perception?.

The answer to this question might be easy to respond by just justifying VR was recently born and a lot has to be done to get the tools we need for an almost real immersion. However, this is not totally true there has been shown through experiments high level of immersion in some VR systems and there are very successful facts in different fields such us medical, entertainment and military training.

These true stories make our problem a bit more difficult to determine why humans, despite of the high levels of immersion, can still realize that a virtual world is not a real one when they are in virtual environments (VEs).

ANALISYS AND DISCUSSION

As we consider factors of immersion as the field to be investigated we firstly need to know what exactly immersion refers to and make a clear difference between related concepts that many people within the VR community confuse, such us presence.

Professor Doug A. Bowman and Ryan P. McMahan in his inspiring paper referenced Mr. PhD. Mel Slater who defines the following important terms:

Immersion refers to the objective level of sensory fidelity a VR system provides.

Presence refers to a user’s subjective psychological response to a VR system.


3

Departing from this point and having clear that both concepts although they are intrinsically related one is objective and the other subjective respectively.

A VR system’s level of immersion depends only on the system’s rendering software and display technology (including all types of sensory displays). Therefore Immersion is measurable so the consequence is that one system can have a higher level of immersion than another.

Presence, on the other hand, is an individual and context-dependent user response, related to the experience of “being there.” Different users can experience different levels of presence with the same VR system, and a single user might experience different levels of presence with the same system at different times, depending on state of mind, recent history, and other aspects.

Figure 1.The human-VE interaction loop.

In the figure above (taken from Bowman’s paper) we see the components of immersion as very technically and it excludes interaction from being part of immersion, his paper holds: “a more realistic method of interacting with the environment does not raise the level of immersion”. I have my doubts about it. The user interface which is the bridge for interaction between human and the VR system must certainly play a very important role while talking in terms of immersion. Perhaps the picture below can clarify the loop a bit better. Notice how user’s goals are transformed while traveling through the loop, at the end the user perceives the information.


4

Figure 2.Human interaction with VR systems.

Since the objective concept immersion is measurable and therefore controllable we can surely determine its factors taking into account they are essentially link to the human body and its sensory systems.

In real life we live in an environment, the real world, which is perceived by our senses in an impressive way. Every time we see, hear, touch etc. we send action potentials (nerve signals) to the brain where the information is processed and allowed us to react and take decisions. To create the perfect immersive VEs is important to fully understand our senses and here might be the cause why we can easily distinguish between the real and the virtual world even though mankind has been researching the human body for so long having a high and accurate knowledge of how our body works.

The most important senses by which humans received information about the world surrounding them are vision, hearing and touch and this is why VR focuses mainly on these senses in terms of research.

Visual immersion factor

Most of the information about the environment surrounding us is gathered by using our eyes. Our visual system processes information in two different ways: conscious (looking at a photograph or reading a book) and preconscious (perceiving light, color depth and movement) processing.

Our eye has an important part called the retina which is a light sensitive layer at the back of the eye. It has photosensitive cells called Rods and Cones which convert incidental light energy into signals carried by the optic nerve.

Rods are responsible for night and peripheral vision, they are unable to detect color and the images they relay are not that clear in contrast cones perform in bright conditions given detailed colored views; there are 3 types of color receptors among them which respond roughly to red, green and blue (RGB) and thanks to that it makes sense to use RGB primary model color in computer graphics.


5

These sorts of findings support VR immersion and allowed us to trick or deceive our brains but we also have to define the limitations of our sense or find out to which extent we are capable of doing something.

The extent of the surroundings that we can view at a single instant is governed by the field of view FOV.

Figure 3.Fields of view

The total horizontal field of vision of both human eyes is about 180O without eye movement. Allowing eye movements to the left or right but without head movements the field of vision increases to 270O. The vertical field of vision is typically more than 120O. This raises the question if we exactly take the same humans’ FOV with both eyes will we have a better immersion? Well recalling the experiment done by Pausch, Proffitt, Williams a decade ago we can clearly answer yes! It definitely affects immersion but we don’t necessarily need the complete FOV, many say that starting from a FOV of 90O you can be immersed.

Another important concept to understand the way humans see is stereoscopic vision in other words three dimensional vision. We are capable of guessing accurately how far an object is from us by just seeing them this is depth perception and stereo vision is considered the main depth cue (monocular depth cues are also present e.g. light and shade ). Here is how it works:


6

Figure 4. Human visual system.

Each eye captures its own picture of the world, this images travel all the way to the back of the brain through the optic nerve, it’s then in that part of the brain called visual cortex where the images are combined into one taking into account the small differences of the pictures, the result is a new picture with depth, thus, a stereoscopic image. (figure 4).

There are of course other important immersion factors within the visual field such us:

• field of regard (FOR)—the total size of the visual field(in degrees of visual angle) surrounding the user,

• display size, • display resolution, • stereoscopy—the display of different images to each

eye to provide an additional depth cue, • head-based rendering—the display of images based on the physical position

and orientation of the user’s head (produced by head tracking), • realism of lighting, • refresh frame rate.

This factors combined carefully affect considerably the level of immersion and in consequence the user’s performance in complex tasks in a VR system according to Bowman and McMahan’s research “How Much Immersion is Enough”, where they compare tasks performance (speed and accuracy) on low-immersion and high-immersion systems. They used the cave to implement both systems.

“The high immersion condition uses the CAVE’s full capabilities, but we use only a single screen—without stereo and head tracking—to produce a low-immersion condition similar


7

to both the desktop PC and the GeoWall. Table 1 shows the level of selected immersion components for the three real-world systems and our study’s two experimental systems.”

The conclusions were seriously convincing and the most relevant for my paper probably was the following: ”We believe that the large displays with wide FOVs facilitated navigation and way-finding in the environment, keeping users from becoming disoriented”…”The combination of wide FOV and high resolution provides a less cluttered, more comprehensible VE.”

Auditory immersion factor

Humans auditory system also have, as in the visual system, cues; Interaural time difference which refers to a measure of the difference in time when a sound enters our left ear and when it enters our right ear, Interaural Intensity Difference which is a measure of how a sound’s intensity level drops off with distance, and Acoustic Shadow which is the effect of higher frequency sounds being blocked by objects between the sound’s source and us.

The way we hear sounds in a simplified manner is the following: our ears guide the sound waves into the auditory canal which in turn enhances the sounds we hear and directs them to the eardrum. The eardrum converts the sound waves into mechanical vibrations. In the middle ear, three tiny bones amplify slight sounds by a factor of 30 then the inner ear changes the mechanical vibrations into electrochemical signals, and these are sent to the brain via the auditory nerve.


8

Figure 5.Auditory System

Although not much research is done in this field as in the visual immersion field In VR systems, I believe 3D sound constitutes an important part in terms of immersion and essentially while talking about “being there” (presence).

At a first glance it might be not that hard to implement it but if we want to take this immersive 3D surround sound factor closer to reality, then we have to take into account head related transfer functions (HRTFs) and this is serious business because the source sound is modified all the way through the air colliding with objects etc. until it reaches our ears which it will be at the end a different sound. HRTFs consider every single aspect while hearing sounds e.g. source sound’s pressure, place where the listener is, the environment the person is, objects in that environment etc.

Karen McMenemy and Stuart Fergusson in his book “A Hitchhiker’s Guide to virtual reality” put the University of Florida as a leader in this field of research.

Somatic Factor

The sense of touch implies more specific details than just touching something. When we touch someone or something we perceive more information (other senses) for example, pressure, shape, textures, temperature, movement and so on. Since this sense of touch involves too many senses we will discuss the haptic1 system which is more limited and only incorporates tactile and kinesthetic information.

So when we get just contact with something (this sense is provided by touch receptors) t we can perceive an object’s shape or texture. However, when we apply more force, kinesthetic information comes into play, providing details about the position and motion of the hand and arm and the forces acting on them, to give a sense of total contact forces and even weight.

1 Haptic, derived from the Greek word haptesthai, meaning to touch, emerged in the 1990s as the term used to describe the sensation of communicating with a computer through the sense of touch.


9

The sense of touch works as follows: there are basically four kinds of sensory organs (fig.6) in the skin of the human hand namely Meissner’s corpuscles, Pacinian corpusles, Merkel’s disks and Ruffini endings.

Figure 6. Sensory organs in a human hand.

Meissner’s corpuscles are found throughout the skin, with a large concentration on fingertips and palms, they are sensitive to touch but cannot detect pain or pressure.

Pacinian corpusles are larger and fewer in number than Meissner’s corpuscles and are found in deeper layers of the skin. They are sensitive to touch and preasure. Merkel’s disks are found in clusters beneath ridges of the fingertips. They are the most sensitive receptors to vibrations. Finally, the Ruffiny endings are slowly adapting receptors that can sense pressure when the skin is stretched.

Although sense of touch is well understood the sense of kinesthesis is more complicated and therefore the haptic interaction between a human and a computer requires a special type of the device that can convert human movements into quantities than can be processed by the computer. The result of the processing should be converted back into meaningful forces or pressure applied to the human as a result of her initial action.

Such devices (input and output) are also a factor immersion and in this field we can answer to our question why it is still a fake environment? By admitting it’s incredibly hard to firstly build a hardware system that will gives us a so many sensori stimuli (recalling that the sense of touch comprehends a lot of more senses) at the same time and secondly the software to receive and process that information and send back to the device.


10

Researchers have been focusing on just few senses (e.g. haptic system) with curious results Kawasaki & Mouri Laboratory from Gifu University in Japan have developed a haptic device that provides kinesthetic sensations called Hiro III which is basically a robot hand that controls force very precisely. To accurately impart a 3D force, it uses compact sensors at the fingertips. Humans have many degrees of freedom so the robot has 15 degrees on the fingertip alone.

Figure 7.Haptic device.

Profesor Kawasaki believes this will have a great impact in different fields, for example, tactile diagnostic training for physicians or tactil screening for breast cancer.

The factor smell and taste are not playing an important role within VR and since there is a lot of work to be done yet in the field it’s also an situation of priorities. Nonetheless there are already some research and even devices that produces sensori stimuli for this specifically senses.

CONCLUSIONS

Immersion is objectively a technical issue in terms of VR systems. We should not confuse it with the feeling of “being there” in other words with presence which is a benefit of immersion.

The visual immersion factors can give us a strong level of immersion if we select, combine and adjust them properly. What is more if immersion is high we will have positive consequences in terms of user’s performance while executing a


11

complex task this is demonstrated by Doug. A. Bowman and Ryan P. McMahan’s experiments, in contrast if immersion is low we can find negative consequences in user’s performance while executing a difficult task as this is confirmed by Pausch, Dennis Proffitt and George Williams in their experiment.

We need more detailed information about; how we work, our capabilities and our limitations then we can implement better systems or even deceive or trick our minds to gain immersion and presence. An clearly example is the RGB primary model color model used by computer graphics, come from the principle that photosensitive cells and the 3 types of color receptors (cones) roughly responds to red green and blue colors.

Detailed information alone it’s not sufficient we must also be able to combine all of it. Researchers work in specifically fields to get something work and that is risky because we might be making a mistake that will cause problems to future generations.

Lack of knowledge or lack of understanding concepts stops us to think clearly and design the requested algorithms to simulate the real fact. For instance the sense of touch as a whole, I believe, we still have a lot to discover and find so that the right algorithms can be design and systems can be implemented concurrently. Nevertheless it’s important to motivate researchers who focus in a specific field and try to push the limits within it like Professor Kawasaki from the Gifu University with his haptic device.

Author Note: VR systems is a booming topic nowadays with its great impact and success we definitely are going to get much better in this field and we will be there faster than we think, thanks to the amount of dedicated researchers and VR community. However, we should think about the past, learn from the big mistakes done with other technologies (The Internet’s negative consequences), and before going on think ethically about the future and the impact it will have in society which one way or another should be the purpose of our findings.


12

REFERENCES

Bowman, D.A., Ryan P. McMahan,”How Much Immersion is Enough”, Virginia Tech, Blacksburg. July 2007 Mel Slater publications: http://presence.stanford.edu:3455/Collaboratory/535 Randy Pausch, Dennis Proffitt, George Williams, “Quantifying Immersion in Virtual Reality”, University of Virginia, 1997; http://www.cs.cmu.edu/~stage3/publications/97/conferences/siggraph/immersion/ Karen McMenemy, Stuart Ferguson, “A hitchhiker’s guide to virtual reality”. copyright 2007 AK Peters Ltd. Bowman, D.A., Ernst Kruijff, Joseph J. La viola, Ivan Poupyrev, “3D User interfaces theory and practice”, copyright 2005 by Pearson Education. Kay M. Stanney, “Handbook of Virtual Environments”, copyright 2002 by Lawrence Erlbaum Associates Inc. Diopsys Educational channel; https://www.youtube.com/watch?v=wbVdlIc5DPE Brian Roberts, “Inaugural lecture”, Professor of Auditory Perception at Aston University; https://www.youtube.com/watch?v=tRWpIUis-x4

Kawasaki&Mouri Laboratory, Gifu University, “3D Haptic System”; https://www.youtube.com/watch?v=XxlYY0xo4gk

Sensory system; https://www.youtube.com/watch?list=HL1357436366&v=TAzTFgPSPiU&NR=1 National Academy of Sciences; http://www.nasonline.org

http://presence.stanford.edu:3455/Collaboratory/535

http://www.cs.cmu.edu/~stage3/publications/97/conferences/siggraph/immersion/

http://www.cs.cmu.edu/~stage3/publications/97/conferences/siggraph/immersion/

https://www.youtube.com/watch?v=wbVdlIc5DPE

https://www.youtube.com/watch?v=tRWpIUis-x4

https://www.youtube.com/watch?v=XxlYY0xo4gk

https://www.youtube.com/watch?list=HL1357436366&v=TAzTFgPSPiU&NR=1

http://www.nasonline.org/

Factors Influencing Virtual Reality Immersion

Documents