Input Device’s, User Interfaces, VR Environment’s TEJASWI CHIRUMALLA U19091105.

Input Device’s, User Interfaces, VR Environment’s

TEJASWI CHIRUMALLA

U19091105

Locator Device

A locator class device returns a position on the display surface selected by the user. It lets the user position a prompt such as a ``+'' on the workstation surface to define a point. The user can then trigger a request for input. It provides the following logical input values to the application.

Example:

Mouse, Keyboard (Cursor-Position Keys), Tablet(Digitizer), Trackballs, Light pens.

Applications:

Interactive drawing and editing, Graph digitizing.

Stroke Device:

A stroke-class device returns a sequence of point positions that the user draws on the display surface. Your application must specify a distance vector and a length of time in the initialization data record.

A stroke-class device provides the following logical input values to the application:

The sequence of WC points.

The normalization transformation number.

The view index (three-dimensional devices only).

Choice Device:A choice-class device lists a series of choices on the display surface. The user selects a choice, and DEC GKS returns an integer value representing the choice.

It provides the following logical input values to the application:The choice statusThe integer representing the choice made

Example:

Mouse, Keyboard (Function Keys), Touch Panel.

Applications:

Interactive menu selection, Program control.

Picking Device:A pick-class device returns the identity of a segment selected by the user and a pick identifier . A pick identifier  is a name that is attached to an output primitive within a segment.   When pick input is requested, DEC GKS provides a cursor on  the screen. When the cursor is in contact with a segment to  be picked and the device is triggered, the cursor provides the  following logical input values:  

The pick status.

The segment name.

The pick identifier  

Example:

Mouse, Cursor Keys, Tablet.

Applications:

Interactive editing and positioning

Physical Devices

mouse trackball light pen

data tablet joy stick space ball

RM

Digitizer

Joystick and Trackball

Pick Correlation:

Pick correlation is the process of determining whether a user-controlled cursor (such as a mouse cursor or touch-point on a touch-screen interface) intersects a given graphical object (such as a shape, line, or curve) drawn on the screen. This testing may be performed on the movement or activation of a mouse or other pointing device

Example:

Three Approaches:

Hit list: Most general approach but most difficult to implement.

Use back or some other buffer to store object ids as the objects are rendered

Rectangular maps: Easy to implement for many applications

See paint program in text

Rendering Modes OpenGL can render in one of three modes selected by

glRenderMode(mode) GL_RENDER: normal rendering to the frame buffer (default)

GL_FEEDBACK: provides list of primitives rendered but no output to the frame buffer

GL_SELECTION: Each primitive in the view volume generates a hit record that is placed in a name stack which can be examined later

Selection Mode Functions and Picking

glSelectBuffer(): specifies name buffer.

glInitNames(): initializes name buffer

glPushName(id): push id on name buffer.

glPopName(): pop top of name buffer.

glLoadName(id): replace top name on buffer.

id is set by application program to identify objects

As we just described it, selection mode won’t work for picking because every primitive in the view volume will generate a hit

Change the viewing parameters so that only those primitives near the cursor are in the altered view volume

Use gluPickMatrix (see text for details)

Picking Template Algorithm

Picking Ideas and getting it to buffer

Scissor Box & Alpha test

Using Regions of the Screen Many applications use a simple rectangular

arrangement of the screen Example: paint/CAD program

Easier to look at mouse position and determine which area of screen it is in than using selection mode picking

Using another buffer and colors for picking

For a small number of objects, we can assign a unique color (often in color index mode) to each object

We then render the scene to a color buffer other than the front buffer so the results of the rendering are not visible

We then get the mouse position and use glReadPixels() to read the color in the buffer we just wrote at the position of the mouse

The returned color gives the id of the object

Writing Modes

XOR write

Usual (default) mode: source replaces destination (d’ = s) Cannot write temporary lines this way because we cannot recover

what was “under” the line in a fast simple way

Exclusive OR mode (XOR) (d’ = d s) x y x =y

Hence, if we use XOR mode to write a line, we can draw it a second time and line is erased!

XOR in OpenGL

There are 16 possible logical operations between two bits

All are supported by OpenGL Must first enable logical operations

glEnable(GL_COLOR_LOGIC_OP)

Choose logical operation

glLogicOp(GL_XOR)

glLogicOp(GL_COPY) (default)

Input Modes:

Input devices contain a trigger which can be used to send a signal to the operating system Button on mouse

Pressing or releasing a key

When triggered, input devices return information (their measure) to the system Mouse returns position information

Keyboard returns ASCII code.

20

Request Mode

Input provided to program only when user triggers the device

Typical of keyboard input Can erase (backspace), edit, correct until enter (return) key (the

trigger) is depressed

21

Event Mode

Most systems have more than one input device, each of which can be triggered at an arbitrary time by a user

Each trigger generates an event whose measure is put in an event queue which can be examined by the user program

22

Rubberbanding

Switch to XOR write mode Draw object

For line can use first mouse click to fix one endpoint and then use motion callback to continuously update the second endpoint

Each time mouse is moved, redraw line which erases it and then draw line from fixed first position to new second position

At end, switch back to normal drawing mode and draw line Works for other objects: rectangles, circles

Rubberband Lines

initial displaydraw line with mouse in XOR mode

mouse moved to new position

first point

second point

original line redrawn with XOR

new line drawn with XOR

25Introduction

What is Virtual Reality(VR)?

Virtual Reality refers to a high-end user interface that involves real-time simulation

and interactions through multiple sensorial channels.

Recent Innovations

Recent innovation from Samsung , Samsung Gera VR powered by Occulus

Display Methodologies:Simulation based VR:

Driving simulators, for example, give the driver on board the impression that he/she is actually driving an actual vehicle by predicting vehicular motion caused by driver input and feeding back corresponding visual, motion, audio and proprioceptive cues to the driver.

The simulator normally consists of several systems as follows:

A real-time vehicle simulation system performing real-time simulation of vehicle dynamics; motion visual and audio systems reproducing vehicle motion driving environment scenes and noise sensed by a driver during driving a control force roading system acting as an interface between the driver and the simulator;

An operator console for monitoring system operation; and system integration managing information and data transfer among subsystems and synchronization. The driving simulators have been used effectively for vehicle system development safety improvement human factor study

Avtar Image Based VR:With avatar image-based virtual reality, people can join the virtual environment in the form of real video as well as an avatar.

The proposed image VR system can handle two types of users.

One can participate in the 3D distributed virtual environment as form of either a conventional avatar or a real video. Background of the video is effectively eliminated to enhance the sense of reality. A user can select his/her own type of participation based on the system capability.

Users with capture board and camera may select a video avatar while others select a conventional computer graphics-based avatar. Avatar image-based VR now provides pretty good interaction environment between human and computer far beyond the conventional desktop computer systems. High-speed networks become available with the advance of network technologies.

Project Based VR In projector-based virtual reality, modeling of the real environment plays a

vital role in various virtual reality applications, such as robot navigation, construction modeling and airplane simulation.

Image based virtual reality system is gaining popularity in computer graphics as well as computer vision communities. The reason is that is it provides more realism by using photo realistic images and the modeling procedure is rather simple. In generating realistic models, it is essential to accurately register acquired 3D data. Usually, camera is used for modeling small objects at a short distance.

Desktop Based VR: Desktop-based virtual reality involves

displaying a 3-dimensional virtual world on a regular desktop display without use of any specialized movement-tracking equipment.

Many modern computer games can be used as an example, using various triggers, responsive characters, and other such interactive devices to make the user feel as though they are in a virtual world. A common criticism of this form of immersion is that there is no sense of peripheral vision, limiting the user's ability to know what is happening around them.

True Immersive Virtual Reality: Hypothetical virtual reality as immersive as consensus reality. Most likely to be

produced using a Brain-computer interface. An intermediate stage may be produced by "Virtual Space" using a head-mounted display with head tracking and computer control of the image presented to the helmet.

VR Examples:Telepresence VR

VR Examples:

Augmented VR

VR Examples:Distributed VR

Technologies of VR Head-Mounted Display (HMD)

- A Helmet or a face mask providing the visual and auditory displays.

- Use LCD or CRT to display stereo images.- May include built-in head-tracker and stereo headphones

Technologies of VR Binocular Omni-Orientation Monitor (BOOM)

- Head-coupled stereoscopic display device.- Uses CRT to provide high-resolution display.- Convenient to use.- Fast and accurate built-in tracking.

Technologies of VRCave Automatic Virtual Environment (CAVE)

- Provides the illusion of immersion by projecting stereo images on the walls and floor of a room-sized cube.

- A head tracking system continuously adjust the stereo projection to the current position of the leading viewer.

Technologies of VR Data Glove

Outfitted with sensors on the fingers as well as an overall position/orientation tracking equipment.

Enables natural interaction with virtual objects by hand gesture recognition.

Technologies of VR Control Devices

Control virtual objects in 3 dimensions.

Architecture of VR System

Input Processor, Simulation Processor, Rendering Processor and World Database.

InputProcessor

RenderingProcessor

World Database

SimulationProcessor

visual, auditory, haptic, touch…

Position &Orientation

Applications

Entertainment

More vivid

Move exciting

More attractive

Applications Medicine

- Practice performing surgery.

- Perform surgery on a remote patient.

- Teach new skills in a safe, controlled environment.

Applications Manufacturing

Easy to modify

Low cost

High efficient

Applications

Education & Training

Driving simulators.

Flight simulators.

Ship simulators.

Tank simulators.

Current problems & Future work

Cybersickness / simulator sickness Low-fidelity Expensive Lack of integration between application packages

High-fidelity system Cost-saving Collaborative High-level contact between participants in distributed

VR

Graphics User Interface:Graphics have revolutionized user interface design. Properly used it can

effectively utilize a person's information processing abilities and allow for faster interaction with computer systems.

Characteristics of a GUI:

A user interface is a collection of techniques and mechanisms that allow a user to interact with a system.

Graphical: primary interaction mechanism is a pointing device

Objects: The user interacts with a collection of elements called objects which are always visible to the user and are used to perform tasks.

Actions: the user performs actions on objects such as accessing and modifying by pointing, selecting and manipulating

User Interface Styles Direct manipulation

GUI (graphical user interface)

WIMP (windows, icons, mouse, pull-down

menus)

Menus

Forms

Command language

Direct Manipulation Interface Characteristics

Screen objects resemble physical objects

Objects arranged in 2-D space

Trades perceptual motor operations for linguistic operations

Use of recognition in place of recall

Expensive to implement

Direct Manipulation Interfaces Allow

Novices to learn basic fundamentals quickly Experts to carry out new tasks Knowledgeable intermittent users to retain operational concepts Error messages are rarely needed Users can assess progress to goals and make changes instantly Users experience less anxiety because systems is understood

and actions are reversible Users gain confidence and mastery through their sense of

control over the system

Direct Manipulation Concerns

Increased system resources

Cumbersome actions

Weak macro techniques

History tracing is hard to log

Visually impaired users at risk

Problems with Direct Manipulation Interfaces

Visual representations are more spread out than simple text - causing "off page" problems

Users must learn meaning of components (e.g. icons) which are meaningful to designer and not user

Visual representation may be misleading

Touch typists do better with a keyboard than with a mouse

Icons The debate concerning text versus icons is an emotional one.

The usefulness of icons depends on how quickly user can figure out their meanings.

Components for a Successful Virtual Reality Application

Visual display

Head positioning and sensing

Hand positioning and sensing

Force feedback

Sound input/output

Other sensation

Cooperative and competitive VR requires networking

Menu Architectures Single

Linear sequence

Tree

Acyclic network

Cyclic network

Menu Item Design Guidelines

Command set small enough to fit within single menu User always have access to all possible menu items

without having to refer to a manual Logical presentation sequences (time, numeric,

alphabetic, physical properties, function/task organization, frequency of use, most important first)

Icons harder to recognize than words for abstract concepts

Avoid screen clutter Don’t assume user will notice subtle cues (e.g. color or

border changes)

Making Selection Easy

Provide command key bypasses for frequent commands

Ensure consistent selection and navigation methods throughout

Be aware of Fitt's law considerations for pointing devices

GOMS Model for Command Language Interface User Task

Step1. Think of and enter command verb

Step2. Think of and enter next argument

Step3. if more arguments then

go to Step2

Step4. if command is incorrect then

correct the command

Step5. Signal computer to process the command.

Step6. go to Step1

Dialog Boxes Combinations of all four interface styles (menu, form fill-in, direct

manipulation, command line)

User task model would also be a composite model

Dialog Box Design Guidelines

Meaningful title

Top-left to bottom right sequencing

Proper clustering and emphasis

Consistent language

Consistent terminology, fonts, capitalization, justification

Standard buttons

Error prevention by direct manipulation

Dialog Box External Relationship Design Guidelines

Smooth appearance and disappearance

Distinguishable boundaries

Sized to reduce overlap problems

Displayed close to appropriate screen objects

No overlap of required items

Easy to make disappear

Clear directions to cancel or complete operations

Advantages of GUIs Symbols are easy to recognize and fast to learn.

Colour is important for classifying objects.

Symbols can aid in problem solving.

Casual users can remember symbols easier than words.

Visual and spatial cues can be utilized to provide more information in a natural way. This, of course, excludes the visually-impaired.

Advantages of GUIs

Some types of error situations can be avoided because they are not allowed to occur through input.

Reduces the need for typing skills. Pointing skills are needed instead and this is not trivial for all types of

users (the elderly, persons with limited fine motor control).

Immediate feedback allows for a better conceptual model of the system for the user.

Advantages of GUIs More attractive than other interfaces and thus encourages more

interaction and exploration. Symbols have the potential to be much more universal than

natural language text. There are still cultural differences in the interpretation of symbols

and colours and thus the need to consider the consequences of internationalism are not entirely eliminated.

There are many fewer symbols than words so not everything can be expressed as a symbol.