A Low-Cost, High-Resolution, Video-Rate/67531/metadc705816/m2/1/high_re… · This paper presents an update of Sandia's development of the Scannerless Range Imager technology and

SfiPB--"?-O%7Orl

A Low-Cost, High-Resolution, Video-Rate

J. T. Sackos, R. 0. Nellums, S. M. Lebien, C. F. Diegert Sandia National Laboratory Albuquerque, New Mexico

J. W. Grantham, T. Monson Air Force Research Laboratory

Eglin AFB, Florida

ABSTRACT

Sandia National Laboratories has developed a unique type of portable low-cost range imaging optical radar (laser radar or LADAR). This innovative sensor is comprised of an active floodlight scene illuminator and an image intensified CCD camera receiver. It is a solid-state device (no moving parts) that offers significant size, performance, reliability, and simplicity advantages over other types of 3-D imaging sensors. This unique flash LADAR is based on low cost, commercially available hardware, and is well suited for many government and commercial uses. This paper presents an update of Sandia's development of the Scannerless Range Imager technology and applications, and discusses the progress that has been made in evolving the sensor into a compact, low, cost, high-resolution, video rate Laser Dynamic Range Imager. Keywords: laser radar, LADAR, imaging optical radar.

1.0 INTRODUCTION

Modern military weapons are being asked to engage a growing array of targets in ways that both reduce the risk to our service personnel and minimize the collateral damage to the civilian populations of our adversaries. As a result, precision strikes are becoming an important aspect of military doctrine. However, in order to achieve the high probability of target kill required from fully autonomous weapons, much more accurate real-time target, guidance, and fuze decisions are required, and this will require further improvements to onboard sensors, target discrimination algorithms, and computational processing.

An imaging optical radar (laser radar or LADAR) is one sensor technology that is uniquely capable of providing the required sensory perception needed to allow a machine to accurately recognize and identify targets or objects in remotely sensed scenes. Imaging optical radar inherently has high spatial resolution and is particularly well suited for the collection of high fidelity, image-quality data. Very robust target discrimination can also be easily achieved with range imagery, and used to exploit the environmentally invariant geometric signature of target objects. Because laser radar is an active sensor, it is less affected by ambient conditions, such as changes in solar illumination and background temperatures, which can cause large target-to-background contrast problems in many passive sensors.

In general, munition guidance can be thought of as a specialized case of a more general robotic vision problem, and the ability to sense and maneuver within an unstructured environment is a basic mobility need of the mobile platform. Therefore, a fast range-imaging sensor has many potential uses beyond weapon delivery, Other such potential applications include: obstacle avoidance, autonomous guidance,

DISCLAIMER

'This rqmt was prepared as an account of work sponsared by an agency of the United States Government Neither the United States Government nor any agency thmof. nor any of their employas maka any warranty, express or impiied. or

fulnm af any information, apparatus, praiuct, or proasr disdoscd, or reprrsents that its IUC would not infringe privately owned rights. Rcfcra~ce herein to any spe- dtic commerciaf product, proats, or service by trade name, trademark, manufac- tnrcr, or otherwise Qes not necesprily coastimtc or imply its adorsanent. fccom- meadation, or favoring by the United States Government or any agency thereof. 'Ibe Vims and Opinions of authors exprrssed herein do not accesdly state or reflect thosc of the United States Government or any aecny thereof.

ustllllcs any legpl liability or responsibility for the ~ c y , . C O m p k e n e s s , or w-

object recognition, automated inspection, vision enhancement, site security and monitoring, terrain mapping, facility or sight surveying, and dynamic structural motion analysis.

2.0 BACKGROUND

Currents imaging optical radar systems use problematic mechanical, solid-state, or natural scanning subsystems. A scannerless system has potentially significant reliability, cost, and size advantages over LADAR sensors that employ mechanical scanning techniques. Because the Scannerless Range Imager (SRI) uses an image intensified video camera as a receiver; it is capable of generating range imagery at video camera rates. Although other types of image intensified video laser radar concepts have been proposed and built in the past, all required either multiple and/or gated receivers, and none provided very good range resolution either because of gate bandwidth limitations, or poor spatial resolution due to pixel density limitations.

Sandia’s Scannerless Range Imager (Figure 1) is a compact, low-cost, high-resolution, high-frame rate, scannerless, range-imaging optical radar. The SRI technology allows fast formation of a range image over a large field of view (object plane) without the use of any type of beam steering or scanning subsystem. It is a floodlight-illuminated, total field-of-view (staring) LADAR system that uses an intensity-modulated light-source transmitter along with an image-intensified charge coupled device (ICCD) video camera receiver. Depending on the desired operating range of the system, either a laser or an array of eye-safe light-emitting diodes (LEDs) may be used as the system transmitter (light illuminator/modulator). Continuous wave (cw), pulsed, quasi-cw semiconductor lasers, and diode or flash lamp pumped crystal lasers have all been demonstrated to be suitable light sources. Both semiconductor lasers and LEDs are low cost and either are or can be made eye safe in certain configurations through spectral wavelength selection and/or spatial distribution of the light at the exit aperture.

Figure 1: Typical Examples of SRI Receivers and a Control / User Interface Computer

The SRI technology is based on conventional cw phase detection electromagnetic radar theory. Real-time numerical extraction of pixel range measurements from the amplitude-modulated scene illumination is made possible by predection mixing of the return signal within an image intensifier and subsequent extraction of the demodulated phase signal using a digital-signal processor. The light source and the image intensifier are synchronously modulated to produce an amplitude variation in the scene illumination and a known modulation in the sensitivity of the camera based receiver. Because the two elements are modulated at the same frequency, a mixing occurs in the image intensifier. The mixing preserves the phase difference between the received laser light that is reflected back from objects in the viewed scene and the modulated image intensifier gain, which acts as a type of injected local oscillator. The mixing and detection of phase occurs simultaneously for each pixel across the entire imaged scene. Because of this predetection mixing, the phase information can be collected and extracted using a very dense array of integrating type detection elements, such as a charge coupled device (CCD) array. The signal processing required to produce a range image requires a few frames of target reflectance data. With the addition of some simple receiver image parallization, the SRI technology also offers the opportunity to capture a full frame of range imagery with each collection of a reflectance image frame. With this enhancement, the SRI concept would stand alone as the only range imager design that could allow an entire high-resolution range image to be achieved with the exposure of a single image from a very inexpensive receiver.

The SRI technology can most easily operate within the visible or near-infrared regions of the electromagnetic spectrum. The bounds of available image intensifier photocathode device sensitivity and the availability of compatible light illuminators define this region. As a result of the electromagnetic wavelength of operation, the SRI has inherently high spatial resolution and is particularly well suited for high fidelity, image-quality data collection. In addition to range imagery, an important feature or consequence of the SRI system is the automatic generation of pixel-registered, actively illuminated photographic imagery. The differential nature of the data acquisition and range imaging processing scheme also helps to minimize any need for stringent light source or detector array uniformity requirements, thereby further reducing system cost.

To date, Sandia has produced several versions of the SRI sensor system using commercially available digital video cameras from NEC, Dalsa, Kodak, SMD, and Cidtec. The camera is connected to a host personal computer (PC) via commercial frame grabbers. The PC provides real-time control of the system and display of the collection reflectance and processed range images. The SRI system is operated through a graphical user software interface program that serves to both control the SRI system and implement the various processing algorithms required to produce range images from collected reflectance images. Convenient data value interrogation, manipulation, and display of the reflectance and range images are achieved with this user-friendly graphical interface and control software program (an example of this interface is shown in Figures 3 and 6). The operating environment software is written in C language and provides a dual-window display with pop-up, push-button menus that respond to point- and-click mouse operations. The software controls the storage, loading, and manipulation of all image data associated with the system and also includes embedded online documentation that allows easy access to many of the menu-driven control features of the system.

3.0 ACCOMPLISHMENTS

Over the past five years, Sandia National Laboratories has been developing the Scannerless Range Imaging hardware, software, and analytical system model to support both field demonstrations of the technology and the ability to accurately predict system performance for specific sensor and

environmental configurations [ 1-91. Because of the success of the development and test activities, the last few years have been very technically gratifying for Sandia’ s Scannerless Range Imager development team, our government and industry technical collaborators and sponsors. In terms of LADAR technology, the Sandia system is presently the only flash laser radar system in the world that has produced inch type resolution, out to kilometer type distances, and sub-inch resolution at shorter ranges. From this flash LADAR innovation, the first strap-down laser radar for munition guidance may evolve (one that could both significantly reduce the size and cost of a LADAR seeker, and significantly increasing the frame rate to video rates and beyond).

In 1996, we completed the initial operational integration of a single Texas Instruments C40 digital signal processor into a SRI system that was constructed around a Dalsa video camera and a Commodore Amiga personal computer (our DalsdAmiga SRI system). This custom sensor design enabled the 256 x 256- pixel density SRI system to operate at a 30-Hz range image update rate. As part of this effort, we had intended to add a fast data storage and real-time visualization capability to complete the development activity. However, because of resource constraints, we terminated further development work on this hardware and instead focused our efforts on demonstrating long range daylight LADAR capability, and bringing online a new PC based SRI system architecture that was specifically tailored for a pulsed laser light source transmitter.

In November of 1996, as part of a joint USAF-Wright Laboratories/US Army Missile Command- sponsored LADAR Polarization Discrimination Tower Test at the Robert F. Russell Measurement facility (RMF) at Redstone Arsenal in Huntsville, Alabama, we were invited to demonstrate long-range daylight operation of the SRI sensor. For this test, we used a highly modified version of the DalsdAmiga SRI system, along with a frequency doubled pulsed crystal laser built by Big Sky Laser Inc. Testing was conducted from the top of a 100-m vertical test tower, and very high-quality (< 1-ft range resolution) images of tactical targets were successfully acquired in daylight conditions, at ranges from 300 meters out to 1 kilometer. Figure 2 (right) illustrates the quality of the range imagery collected from the DalsdAmiga SRI LADAR system.

Figure 2: A Photograph and a 1-foot Iso-Range Contoured Range Image from the 256 x 256 Pixel Density DalsdAmiga SRI LADAR System

Having successfully completed the long-range daylight demonstration, we then focused on building the new PC based SRI system. This system was constructed around a Kodak Mega Sensor camera that provides one million pixels per image and 10-bit digitizing resolution, a GEN I11 image intensifier, and a 200 MHz Intel Pentium Processor. This architecture used commercial components and provided a significant processing improvement over the DalsdAmiga SRI system. In support of this transition, we also ported our entire graphical user interface SRI software control program to the IBM PC Windows environment.

Figure 3: SRI Reflectance Image (left) and Gray Scaled Psudo-colored Range Image (right) of the LADAR Calibration Target from a Range of 500 meters.

Figure 4: SRI Range Image of Military Trucks at -300m Range

Figure 5: A Photograph and SRI Range Image of a M60 Tank at -300 meters Range

Figure 6: SRI Reflectance and Range Imagery a 2-Ton Truck at -1 Kilometer

In February of 1997, we were invited by the Air Force to conduct a SRI LADAR sensor calibration and performance assessment test. For this test, we transported our new Kodak/PC SRI system to the USAF Wright Laboratories Laser Radar Development and Evaluation Research Facility (LDERF) located at Eglin AFB, FL. There we successfully range imaged, in daylight conditions, large-resolution targets (8.8 m x 4.8 m) at a range of 500 meters. We demonstrated individual pixel resolution (maximum deviation) of better than 6 inches and a fitted plane separation resolution of less than 0.5-inch [lo]. A photograph of the target calibration panel and an example of the collected reflectance and range images from the SRI sensor is shown in Figure 3.

In May of 1997, we were again invited by the Army at Redstone Arsenal to use their Robert F. Russell Measurement Facility (RMF) to further demonstrate improved daylight LADAR operation using our new higher resolution Kodak/PC SRI system. In this test we demonstrated better than 6-inch range resolution on various types of tactical military vehicles. An example of the type of range imagery collected at this test is illustrated in Figure 4.

In September of 1997, a very ambitious series of field-tests was undertaken. At the request of the Air Force, we began the series of LADAR field tests by participating in a LADAR sensor Bomb Damage Assessment (BDA) test at Eglin AFB. Using the SRI LADAR, a target was imaged both before and after an explosive warhead test, and from the collected LADAR data, the physical extent of explosive damage to the target was successfully detected [ 113.

After completing the BDA test with the Air Force, we again traveled to the Robert F. Russell Measurement facility at Redstone Arsenal to conduct some additional long-range daylight LADAR testing using our new higher resolution, KodaHIBM PC SRI system. In this collaborative test with both the Army and Air Force, we operated the SRI LADAR system for the first time with a 250 mJ per pulse, 10 Hz, frequency doubled crystal laser. Figures 7 and 8 illustrate the type of range image data that was collected at this test on tactical mobile military targets (an M60 Tank and a 2-Ton Truck) at ranges from 300 meters to 1 kilometer.

In collaboration with NASA’s Ames Research Center (NASA-ARC) and Johnson Space Center (NASA- JSC), Sandia National Laboratories has also been working on applying SRI technology to potentially meet a number of NASA interests in ranging imaging sensors. NASA applications include: autonomous rendezvous and docking, robotic manipulator location, high-resolution terrain mapping, autonomous spacecraft landing, mobile robotic vehicle mobility, and the vibration analysis of large space structures.

For mobile robotic vehicle mobility, a very small Scannerless Range Imager was developed for both evaluation by NASA-ARC and for use on a Surveillance And Reconnaissance Ground Equipment (SARGE) robotic vehicle that was recently developed by Sandia for the US Department of Defense [ 121. Figure 7 illustrates the SRI sensor for the SARGE robotic vehicle.

For vibration analysis of large space structures, a very high resolution, video rate Scannerless Range Imager was developed for NASA-JSC. Figure 8 illustrates the configuration of the SRI sensor that was used in this initial proof-of-concept ground based field demonstration. This test was intended demonstrate the ability of the SRI technology to accurately recover the vibration modes of large vibrating structures (0. l-inch peak-to-peak displacements at ?A Hz vibration frequencies). Figure 9 shows the sensor setup at NASA - Johnson Space Center and some of the targets used in the testing. Figures 10 and 11 show the vibration data results from two tests.

Figure 7: A SRI Sensor for Mobile Robotic Vehicles

Figure 8: A High Resolution, Video Rate SRI Sensor for Structural Dynamics

The continuous wave transmitter for this NASA-JSC test consisted of a 5W, 800-nm laser diode with near 100% depth of modulation at 140 MHz. A 5-degree light diffuser on the transmitter yielded a field of illumination of approximately 6 feet at the target. The receiver consisted of a standard GEN I11 image intensifier with an estimated 510% depth of modulation at 140 MHz. An 85-mm lens on the receiver

yielded a field of view of about 9 degrees or 10 feet at the target. Lens speed was F1.8. The intensifier was coupled to a 512 x 512 CCD camera that was capable of operating at up to 120 frames per second. In each test, a total of 800 images were captured (typically at a 15 Hz image capture rate). A range image was calculated from every four consecutive reflectance images.

The “radiator” target (Figure 9 (middle-right), and data results shown in Figure 10) consisted of a rigid flat plate with diffuse white coating. The surface was normal to the SRI viewing angle, and a hydraulic actuator normal to the surface in the range axis induced sinusoidal rigid-body motion (0.1-inch peal-to- peak, Vi Hz). Range to this target was 70 feet. A “bending plate” target (not shown, but data results shown in Figure 11) consisted of a flexible flat plate that was rigidly anchored at one edge. It also had a diffuse white coating and the surface was again normal the SRI viewing angle. Natural modes in this structure were manually induced by initially displacing the edge of the plate (opposite the anchored edge) by a few inched and then releasing it. Range to this target was slightly less than 70 feet.

Figure 9: Illustration of the Test Configuration and Targets

The spectral processing of the results presented in Figures 10 and 11 first included the selection of a region of interest from the full frame of collected SRI range imagery, from which a spatially averaged time series of range pixels was calculated. This was followed by the use of a windowing and spectral decomposition spreadsheet to generate the vibration spectrum from the resulting range time series. The windows consisted of a half sine function over the period of the time series and calculated the amplitude at each frequency using standard Fourier integrals. The results from these tests demonstrated the feasibility of the SRI technology in meeting 0.100-inch peak-peak vibration sensitivity at 70-foot range, and showed a potential path for meeting a sensitivity of 0.010-inch peak-to-peak.

-~ ~ __ __ - .. _. . __ - - ..

Bob-100 Test Results I 32x32 Pixel Average

0.2 0.1 5

i 0.1 I

I .I 0.05 Q) 0

-0.05 pl

-0.1 -0.1 5

Y

m

-0.2 0 10 20 30 40 50 60

Time (Seconds) . .... - ... . _. ... ....... . . . . . . _ _

. - ..... . ..... . _. .. ._ ... -

0.1 4 0.1 2

0.1 0.08 0.06 0.04 0.02

0 0

BOB-100 Test Spectrum 32x32 Pixel Average

. -

0.5 1 I .5

Frequency (Hz)

2

_. -. . -. _ _ .... ..........

I

Figure 10: Radiator Target: Displacement Time History and Vibration Spectrum

__ -. - - __ _--__ .

PLATE Test Results 68x68 Pixel Average

2 1.5 -

1

-1 -1.5 .!

-2 Time (Seconds)

1 2z n 0.8 2i n c 0.6 .- Y

0.4 E a 0.2 5

.- w

...... - . . . . . _ _

PLATE Test Spectrum 68x68 Pixel Average

.. .. .

0 0 0.5 1

Frequency (Hz)

2

. . . . . . ...

Figure 11: Bending Plate Target: Displacement Time History and Vibration Spectrum

I Two 30 Renderded Images of Range Data taken from a Single SRI Range Data Collection

I

Figure 12: Example of High Resolution Range Imagery

Finally, Figure 12 illustrates the type of range resolution that was achieved with the high resolution SMDPC SRI system. This LADAR data set was taken in the laboratory under ideal conditions, and at a range of approximately 20 feet.

The application scope for the SRI development has also include a desire to investigate the ability of this sensor to meet some of the operational needs of US Special Forces divers for mine detection and identification. For this application, the goal for a SRI sensor would be to help identify partially buried or moored mines in shallow water. The sensor would have to be either hand portable or small enough to be integrated onto an underwater manned or unmanned vehicle. It would also have to deliver high quality images in turbid coastal water, be compatible with the size and power constraints imposed by the intended deployment platform, provide dayhight operation, and be able to operate in a manner that minimizes the risk of detection by hostile observers on the surface. Many of the SRI development activities discussed above are synergistic with the development of an underwater sensor, such as achieving higher range and spatial resolutions, incorporating improved sensitivity using a GEN I11 image intensifier, demonstrating operation using a 532 nm green light laser light source.

As part of the field testing conducted at the LDERF at Eglin AFB, we were asked by researchers at the Naval Surface Warfare Center - Coastal Systems Station to collect a set of LADAR images of surface land mines that were dispersed on various types of natural earth type background environments. This active imaging data collection was intended to augment the Navy’s ongoing target discrimination phenomenology testing that they were conducting using several types of passive multi-spectral imaging

sensors. For this data collection, we used a pulsed 800-nm GaAs laser diode illuminator with the Kodak/PC SRI system. Range imagery was collected at dusk, from a standoff distance of 50 feet. Figure 13 illustrates representative range and reflectance imagery collected on a matrix of nine land mines, each positioned on the surface of a different type of natural earth background. Because of the differences in reflectivity between the mines and backgrounds, many of the mines are very apparent in the reflectance imagery, Although only slightly visible in the displayed range image, the several inch geometric profile of the mine above the surface is clearly evident in many of the cases. Therefore, although this SRI LADAR was not optimized for this application, it has successfully demonstrated a utility in being able to aid in remotely sensing unburied land mine and we expect to conduct a actual underwater demonstration of the SRI technology in April of 1998.

Figure 9: SRI Range and Reflectance Imagery of Unburied Land Mines

4.0 FUTURE WORK

Sandia’s principal SRI sensor development efforts have focused on applying this technology to military munition guidance applications and close range underwater mine identification. We have successfully achieved state-of-the-art LADAR performance in many areas, most notably range resolution, image acquisition rate, and image pixel density. We have now demonstrated inch type spatial and range LADAR resolution in both day and night conditions from target distances out to 1 kilometer.

As a result of the many successful development and demonstrations of the SRI technology to date, the SRI technology is now being actively transferred to several U.S. Industries for further commercialization. Sandia National Laboratories is currently working with Eastman Kodak, Insitec Measurement Systems, and the American Iron and Steel Institute (AISI) on a variety of potential

commercial applications for the SRI technology. Sandia has also recently completed a Phase I Research and Development effort with Visidyne, Inc., in which optical range imaging concepts were matured for a counter sniper application [ 131.

In order eliminate the need for acquiring several sequential frames of reflectance data in which changes between successive frames (due to platfordtarget motion) could cause significant errors in the range image if left uncorrected, we are investigating novel approaches to collecting all necessary imagery in a single camera exposure. Second, because of the spectral limitations of current image intensifiers, the most optimum long range daylight pulsed type solid-state transmitter is today a doubled Nd:YAG laser (a green light laser operating at 532 nm). For covertness, improved eye safety and weather penetration, and the availability of higher energy output, use of Nd YAG lasers operating at 1.06 um and 1.54 um are being actively investigated.

5.0 REFERENCES

1. Sandia National Laboratories, “Technology Transfer Opportunity - Scannerless Range Imaging System,” Commerce Business Daily, September 30, 1994.

2. M. W. Scott, Range Imaging Laser Radar, US. Patent 4,935,616, June 19, 1990.

3. J. P. Anthes, P. Garcia, et al., “Non-scanned LADAR Imaging and Applications,” Applied Laser Radar Technology, Proceedings of SPIE, Vol1936, 1993.

4. P. Garcia, J. P. Anthes, et al., “Characterization of a Scannerless LADAR System,” Applied Laser Radar Technology, Proceedings of SPIE, Vol 1936, 1993.

5. M. M. Lecavalier, et al., “Scannerless Range Imaging with a Square Wave,” Guidance and Navigation - Applied Laser Radar Technology 11, Proceedings of SPIE, 1995.

6. Ken Frazier, “Innovative Range Imager Sees How Targets Measure Up,” Sundiu Laboratory News, Vol. 46, No. 19, September 16, 1994.

7. D. H. Cress and M. M. Lecavalier, “Fusion of LADAR with SAR for Precision Strike,”-Proceedings from the Eighth National Symposium on Sensor Fusion, 1995.

8. J. Sackos, B. Bradley, B. Nellums, and C. Diegert, “The Emerging Versatility of a Scannerless Range Imager,” Laser Radar Technology and Applications Conference, SPIE AeroSense Symposium (Orlando, EL), paper #2748-4, (April 1996).

9. J. Sackos, B. Bradley, C. Diegert, P. Ma and C. Gary, “Scannerless Terrain Mapper,” SPIE International Symposium on Optical Science, Engineering, and Instrumentation, Denver, CO, August 1996.

10. C. Diegert, J. Sackos, and R. Nellims, “Building Accurate Geometric Models from Abundant Range Imaging Information,” Laser Radar Technology and Application Conference, SPIE Aero Sense Symposium, Orlando, FL, April 1997.

11. Capt. M. Keltos, “Demonstraion of Imaging LADAR for Battle Damage Indication”, 1998 Meeting of the IRIS Specality Group on Active Systems, Albuquerque, NM, March 1998.

12. J.B. Pletta, J.T. Sackos, “An Advanced Unmanned Vehicle for Remote Applications”, Sandia Report, SAND98-0562, March 1998.

13. 0. Shepherd, L LePage, G. Wyntjes, T. Zehnpfenning, J. Sackos, R. Nellums, “Counter Sniper 3-D Laser Radar Phase I STTR Final Technical Report”, DARPA Contract No. DAAHO1-96-C-R199, September 16, 1997.

M98004665 I11111111 Ill 11111 11111 11111 11111 11111 lllll11111 Ill1 1111

Publ. Date (11)

Sponsor Code (1 8) U C Category (1 9)

DOE