-
Field-of-View Enhancement for NADS Non-Standard Applications
Yefei He, Chris Schwarz, Jeff Gordon, Shawn Allen, Tim
Hanna*
National Advanced Driving Simulator 2401 Oakdale Blvd Iowa City,
IA 52242
*Olin College
Olin Way Needham, MA 02492
Abstract
The National Advanced Driving Simulator was designed as a very
high fidelity device targeted for passenger cars and trucks. The
use of the NADS to simulate a different kind of vehicle would be
classified as a ‘non-standard’ application, in light of its
charter. Two recent examples of non-standard NADS applications are
agricultural and construction vehicle simulation. In both of these
cases, an extended vertical field-of-view (FOV) was required to
achieve the desired level of realism. Fortunately, a solution was
conceived that could address the need of both of these
applications. Two additional projectors were mounted near the
ceiling of the dome oriented downward. Two new screens were
constructed and mounted to the floor of the dome in the locations
where extra FOV was required. This paper describes the design and
implementation of the FOV enhancement for the NADS. Constraints on
space, combined with minimum required image size made the design
challenging. Additionally, channel configuration for the additional
display screens had to be computed for the image generator. Details
on the constructions of low cost custom screens are given, as are
the specifications of the selected projectors. The procedure for
configuring the additional channels is described. Finally, the
application of the FOV enhancement to agricultural and construction
vehicle simulation is reported.
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
Introduction
The National Advanced Driving Simulator (NADS) at the University
of Iowa is a very high fidelity simulator that provides a
convincing driving experience through its immersive virtual
environment that combines audio, visual, motion, and haptic cues.
All activities take place inside the 7.3-m (24-ft) dome where a
full or partial cab of an actual vehicle is placed in the center of
the floor, surrounded by floor-to-ceiling curved projection screens
providing 360° horizontal field of view (FOV) and 40° vertical FOV,
with a slight downward elevation of about -1.2° in the front. Such
configuration is sufficient for reproducing the full view that a
driver observes while sitting inside a regular passenger vehicle,
due to the location of car windows and the inherent view
obstruction caused by the front hood. In addition, during regular
driving, the driver’s eyes usually focus at some distances ahead,
therefore a near-field view of the ground area immediately
surrounding the vehicle is not critical. Many studies carried out
at NADS use regular passenger vehicle cabs, where the existing
display environment is sufficient. On the other hand, the NADS
system is capable of simulating non-standard vehicles with cab
layouts that are very different from passenger vehicles, and with
driving tasks that are best simulated with a non-conventional field
of view, as well. Two NADS applications in particular called for
enhancement to the field of view: simulation of a Caterpillar 980G
medium wheel loader, and simulation of a John Deere 7920 tractor.
Both vehicles have cabs with front and side windows that extend to
the cab floor to provide very good near-field front and side
visibility necessary to operate the vehicles with the requisite
precision for their routine tasks. For the wheel loader, due to the
articulated steering, the front assembly including the bucket, lift
arms and front wheels, can sweep an angle of about 40 degrees
horizontally. The front assembly is not part of the cab, and the
existing display environment cannot cover it at all, leaving a
large gap between the view of the front displays and the cab,
making it difficult for very common wheel loader tasks such as lane
following, lane changing, obstacle avoidance, and truck loading.
Similarly, the gap between the front projection screen and the
tractor cab, although partially filled by the hood which is part of
the cab, still leaves the front wheels and the ground underneath
absent from the virtual environment, which are fully visible in an
actual tractor and are used as a critical visual cue for the
drivers, thus hampering tasks that require precise steering
control. In order to overcome the limited vertical field of view of
the existing NADS display environment and make it viable for
non-traditional applications such as the wheel loader and the
tractor, a plan for the enhancement of the near-field vertical
field of view was developed by using additional projectors and
ground-level projection screens. Emphasis was placed on minimizing
costs while working within the NADS image generator capacity and
channel configuration. Because cab changes are frequent and
necessary, ease of installation, removal and flexible configuration
were important considerations. The solution also needed to fulfill
somewhat different enhanced FOV requirements of the
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
wheel loader and tractor cabs. The plan was implemented and put
to use in several studies, and the solution proved to be
effective.
Design
The NADS-1 dome visual subsystem consists of a computer image
generator (IG) based on the Quantum3D© Independence™ architecture.
This architecture uses individual renderers for different visual
channels. The existing 360° surround view is made up from eight
channels, with images projected from eight ceiling-mounted Barco©
SIM-6 Ultra™ LCD projectors. However, the IG hardware has
additional renderers as spares that are not for use within the
NADS-1 dome. Therefore, it is feasible to utilize the spare
renderers and configure them to render additional channels that
enhance the field of view. The hardware required to complete the
enhanced configuration are additional projectors and screens, and
mounting devices inside the dome that can sustain motion
stresses.
Screen design
Ideal layouts for field-of-view enhancement Based on the
geometry of the dome, the dimensions of the cabs, and the area of
visual coverage to be expanded that the applications demand, two
ideal layouts for near-field ground view expansion were conceived
for the wheel loader and the tractor respectively. The maximum
number of additional channels was limited to four, due to the
number of spare renderers available from the IG. When issues of
cost, ease of installation and removal, and flexibility were taken
into consideration, a two-projector solution was selected, and a
screen aspect ratio of 4×3 was chosen over 16×9. Moreover, in order
to maximize the screen sizes, it was decided that the front
vibration actuators of the wheel loader cab would be removed and
the aluminum frame that secures the cab to the NADS-1 dome floor
would be modified. No such modifications were necessary for the
tractor cab.
Figure 1. John Deere 7920 tractor ideal FOV enhancement
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
Figure 2. Caterpillar 980G Medium Wheel Loader ideal FOV
enhancement
Figure 1 and Figure 2 show the top-down and side views of the
ideal design for the field-of-view enhancement for the tractor and
wheel loader respectively. For the wheel loader, the additional
channels, when combined with the existing channels, provide
complete coverage of the front assembly when the articulation angle
is small, and provide very good coverage even when the articulation
angle reaches its limit of +/-40 degrees. For the tractor, the
additional channels provide complete coverage of the front wheels
regardless of their steering angles. The gap between the main front
channels and the ground view channels shown in the top-down views
is in fact invisible from the driver’s eye point due to the
elevation of the ground view screens, as illustrated by the side
views.
Actual sizing and shape Figures 1 and 2 represent the ideal
layouts for the FOV enhancement, but in reality, some modifications
to the shape and size of the screens were needed. To determine the
size of the viewing area it was necessary to establish the distance
from the center of the dome to the base of the wall and the height
of the observer. The distance from the center of the dome to the
base of the front wall was taken to be 115 inches. The height of
the driver was found by measuring from the ground to the seat level
and then adding the assumed height of the driver. For the purpose
of the design the height to the seat of the tractor cab was 17
inches and 14 inches for the wheel loader cab; the height of the
driver was assumed to be 54 inches for both cabs. Each cab would
also be 11 inches off the ground. Once this was done a simple
trigonometric relationship could be set up to determine the length
of the screen. With the trig diagram in Figure 3, the distance from
the center of the dome to where the observer’s field of view met
with the bottom front wall of the dome was calculated to be 96
inches. This measurement would become the radius of the arc sweep
which formed the basic shape of the screens as can be seen in the
right of Figure 3. To get the actual distance from the screen to
the front of the dome, the distance from the center of the screen
to the front end of the tractor cab was subtracted from the 96
inches. This yielded a screen length of 60 inches. The curved shape
of the screens was intended to allow them to cover the full field
of view from the front of the cab up to the dome’s
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
existing field of view, while minimizing overlap. In addition,
the screens were designed in three sections so that the center
could be removed, allowing the screens to fit on either side of the
tractor body. For the wheel loader cab, the combination of the
three screens is intended to provide full coverage of the front
field of view. This meant that a minimum number of pieces were
needed in order to provide full coverage for the two different cab
designs, reducing the number of elements to fabricate and the
amount of effort required to switch cabs.
Figure 3. Left, trig diagram used to calculate screen
dimensions; right, diagram of screen
Screen fabrication The fabrication design of the screens took
into account both ease of construction and durability. The
construction of the screens was a skeleton frame which would
support a ¼ inch plywood facing covered with stretched muslin and
coated with commercial screen paint. For durability, all three
frames were made from welded steel angle iron. The steel reduces
warping of the plywood frame thus distorting projected images. The
plywood for the frame was cut to the shape of screen and attached
to the frame by screwing the plywood to 1×3" boards through
predrilled holes in the steel frame. Muslin canvas was then
stretched over the frames and coated with a mixture of latex paint,
water and wood glue to help bond it to the plywood. This treatment
helped to provide a smooth, uniform surface upon which to apply the
screen paint. When dried, the screen paint was painted on the
screens.
Projector selection The design of the screen required that the
projectors needed to produce images with an aspect ratio of 4×3.
Also, the cost of each projector was confined to about $2000. Other
factors included ability for image morphing and color adjustments
to match the main channels, weight, and sustainability to motion.
At this price range, the candidates were presentation projectors
and vertical keystone adjustment was the only image morphing
available. Image size, resolution, and brightness also needed to be
examined.
Image size The image magnification m of a projection system is
approximated by
fDm ≈
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
Where D is the throw distance, and f is the lens focal length
([4]). Therefore, the diagonal image size S is
fKD≈mKS =
Where K is the diagonal size of the display element of the
projector. Due to constraints from the dome geometry, the maximum
throw distance from the mounted projector to the screen was only
around 100 inches. In order to achieve the designed image size of
74"×56", or a diagonal of 93", the value of K/f needed to be
sufficiently large. Among the candidates, only the Casio© XJ-360
DLP™ projector, with a 0.7" display element and a zoom lens of
focal length from 20.0 to 40.0mm, met the requirement.
Image resolution Each of the eight main channels of the NADS has
a field of view of 51°×40° including 3° on both sides for edge
blending. The front channels have a pixel resolution of 1600×1200,
while the side channels have a resolution of 1280×1024, and the
rear even less ([1]). In terms of pixels per arc minute, the
resolutions are 0.52×0.50 and 0.42×0.43, respectively. The wheel
loader set up had an image size of 74"×56" per channel. Using the
position information from Figure 3, the FOV was calculated as
approximately 44°×36°. The resolution of the Casio projector’s
display element is 1024×768, making it 0.39×0.36 pixels per arc
minute. For the tractor set up, the image size was 60"×45" with an
FOV of 38°×30°, resulting in a resolution of 0.45×0.43. Thus, the
ground view channels did not have as good resolution as the front
channels, but were comparable to the side and rear.
Image brightness The luminance or brightness B of a projected
image can be calculated by
gSFB =
Where F is the light output of the projector, S is the area of
the projected image, and g is the gain of the screen ([2], [4]).
The Barco projectors have an on-axis light output of 3000 lumens
(lm), and the image size is 12.5ft×8.6ft, resulting in a brightness
of (27.9g) foot-lamberts (fL). For the Casio projector, the on-axis
light output is 2200 lumens with the zoom ratio set at 1×, when the
relative aperture N0 is 2. In the worst case scenario when the zoom
ratio is 2× to achieve maximum image size, the relative aperture N1
is 2.8, and the light output becomes
lmN
FF 11222220022
0 ≈×==N 8.2 221
01 ([4])
The ground view image size for the wheel loader setup was
6.17ft×4.67ft, so the image brightness would be no worse than
(38.9g) fL. The gain of the main channel screen is fairly similar
to that of the ground view screens, which is in the range of 1.8 on
axis. Therefore, the brightness of the ground view images would be
more than sufficient.
Visual channel configuration Mantis ™, the rendering software
that the Quantum3D IG uses, comes with a visual channel
configuration utility tool ([3]). To set up a channel, the size,
position and orientation of the display need to be specified. The
distance for the main channel displays
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
screen.
has been previously set to 0.5 database units (dbu),
corresponding to the 12-ft radius of the NADS dome. The size and
position values of the ground view channel displays would then also
be converted to dbu’s. In the tractor cab setup, the two projectors
were rotated +90° and -90°, respectively. This had to be reflected
in the roll values of the displays. Figure 4 shows the Mantis GUI
for channel configuration and the layout of all displays, including
the 8 main displays, the ground view displays, and a bird’s eye
view
Figure 4. Ground view channel configuration in the Mantis
software. Left, MWL, right, tractor.
Installation
d
ed
achment to the side screens by bolts that d and steel frame.
b
trade
Screen installation Before the installation of the screens in
the dome the frame corners were reinforced anlegs were added to
elevate the screens. The reinforcement of the corners was done by
gluing wood braces into each corner. The legs were short lengths of
aluminum angle with a small aluminum angle foot for attachment to
the dome floor. The legs were mounted tothe screens with bolts that
pass through the wood and steel frame. The small aluminum angle
foot on each leg was bolted directly to the dome floor. The dome
floor was drilland tapped for the mounting of these feet. Only the
side screens had legs added. The center screen was supported by its
direct attpassed through the woo
Projector installation The differing fields of view between the
wheel loader and tractor cabs necessitated two different projector
mounting techniques. The wheel loader cab used the center screen
and required an image at the front of the cab that was wider than
it was deep. The tractor cadid not use the center screen and
required an image on both sides of the hood that was deeper than it
was wide. For the wheel loader, the two projectors were mounted in
the same plane, pointing straight down with no rotation about its
vertical axis. They were installed in the dome by the use of a ball
and socket mounting system sold under the name RAM Mount. This
mounting system was supplemented by metal strapping to
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
e by
ounted projector in e MWL set up with the RAM Mount and metal
strapping visible.
further secure the projectors. This assembly was then attached
to the existing projector framing. For the tractor, the two
projectors were mounted in two parallel planes, pointingstraight
down and rotated 90° about its vertical axis.They were installed in
the dommounting them to two aluminum plates, which were then
attached to the existing projector framing. Figure 5 shows the two
different set ups, and a mth
Figure 5. Ground view set up with MWL (left, screens, center, a
projector); right, with the tractor.
he
be
ere made but they still did not match perfectly. Image
brightness was also reduced.
Results
cement
ks
ly how the continuous isual flow was preserved by the added
ground view channels.
ll ages
Image adjustments After the ground view screens and the
projectors were installed, the two new visual channels were
configured using the Quantum3D Mantis software. Due to the small
throwdistance, the zoom ratio of the projectors was set to 2×. The
image size was measured at 63"×47" for the tractor set up, slightly
exceeding the designed size, but only 66.5"×48.5" for the MWL,
smaller than designed. There were also some keystone effects
because tprojectors were not pointed exactly perpendicularly
towards the screens. The vertical keystone adjustment function on
the projectors could only partially overcome the effects. The
actual size and location of the images were measured, compensated
for the keystone effects, and then used for the configuration.
Several iterations of further adjustments weremade after visual
inspection from the designed driver’s eye point, which turned out
to7 inches behind the center of dome for the MWL. Default color
balance of the Casio projectors was quite different from the Barco
ones. Some adjustments w
As mentioned, the installation and final ground view images
somewhat deviated from the design. However, drivers’ feedbacks from
the studies that utilized the FOV enhanwere positive. Imperfections
in areas such as parallax error, color and brightness matching were
not commented upon by the drivers. Drivers were able to perform
tasthat rely on near-field views in the real world without
complaints. Prior to the FOV enhancement, drivers in similar
studies had complained about a lack of continuation of visual flow
from the front view to the cab. Figure 6 shows clearv A point of
concern had been how rigid the projector and screen mounts would
be, and how they would stand up to the rigorous motion that the
NADS-1 dome is subject to, in particular during the wheel loader
tasks. Results showed the motion seemingly had no ieffect on the
projectors, and there was no visible vibration on the ground view
im
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
ze nd location of the images changed slightly, requiring
adjustments in channel layout.
during simulation. However, gradual drifting took place, and the
visual channel configuration had to be tweaked between simulator
runs to compensate for it. Also, each time the ground view
projectors and screens were reinstalled after a cab change, the
sia
ure 6. Comparison of forward views from inside the actual cabs
vs. the enhanced simulator cabFig s. Top row, Caterpillar 980G MWL,
actual vs. simulated; bottom row, John Deere 7920 tractor.
al
r the drivers evidently did not pay attention to this issue as
no one complained about it.
Conclusions
. In
should take into consideration whether to include near-field
views from the start.
References
m Dr. s.htm
Due to differences in drivers’ heights and seat position
settings, variations in the actueye point location took place.
Drivers also moved their heads when driving, further changing the
eye point. The channel configuration was done to a set eye point
location, which would cause parallax errors when the actual eye
point moved away. Howeve
The field-of-view enhancement increased the range of
applications for the NADS simulator, making it suitable for tasks
requiring near-field ground views. The drivers’ feedbacks in
related studies confirmed the effectiveness of the enhancement.
There were issues when the design was implemented, however they did
not diminish the benefitsthe future, the sturdiness of the mounting
can be improved, and a systematic way of compensating for eye point
variations is desirable. Future designs of driving simulators
[1] Allen, S. (2006). NADS Visual Subsystem. Technical Report.
Iowa City, IA. [2] Calvert, J. B. (2003, August 14). Illumination.
Retrieved March 10, 2006, fro
James B. Calvert's web site:
http://www.du.edu/~jcalvert/optics/lumen[3] Quantum3D, Inc. Mantis
Users Guide for Version 2.0. San Jose, CA.
-
DSC 2007 North America – Iowa City – September 2007
__________________________________________________________________________________________
[4] Ray, S. F. (1988). Applied Photographic Optics: Imaging
Systems for Photography, Film and Video. London & Boston: Focal
Press.
DSC07_FOV_paperIntroductionDesignScreen designIdeal layouts for
field-of-view enhancementActual sizing and shapeScreen
fabrication
Projector selectionImage sizeImage resolutionImage
brightness
Visual channel configuration
InstallationScreen installationProjector installationImage
adjustments
ResultsConclusionsReferences