Drone Based Aerial Imaging For Post-Disaster Reconnaissance Théau Héral 1 , William Greenwood 2 , Dimitrios Zekkos 2 , PhD, PE and Jerome Lynch 2 , PhD 1 Department of Aerospace Engineering • 2 Department of Civil and Environmental Engineering • University of Michigan • Ann Arbor, MI Email: [email protected] • [email protected] • [email protected] References 1. Alaska Dispatch News, (4/19/2015), http://www.adn.com/ 2. The Atlantic, (4/19/2015), http://www.theatlantic.com/magazine/archive/2010/12/the-drone- wars/308304/ 3. Baiocchi, V., Dominici, D., & Mormile, M. (2013). UAV application in post-seismic environment. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-1 W, 2, 21-25. 4. Cinehawk, (4/19/2015), http://cinehawk.co.uk/blog/drone-filming-uk/ 5. Direct Relief, (4/19/2015), https://www.directrelief.org/2013/12/civil-drones-improve-humanitarian- response-philippines/ 6. DJI, (4/19/2015), http://www.dji.com/ 7. Factor, (4/19/2015), http://factor-tech.com/drones/7363-delivery-drones-closer-to-reality-with-self- monitoring-quadcopters/ 8. Huang Y B, Thomson S J, Hoffmann W C, Lan Y B, Fritz B K. Development and prospect of unmanned aerial vehicle technologies for agricultural production management. Int J Agric & Biol Eng,2013;6(3):1-10. 9. MatLab Documentation, (4/19/2015), http://www.mathworks.com/help/matlab/ 10. Remondino, F., Barazzetti, L., Nex, F., Scaioni, M., & Sarazzi, D. (2011). UAV photogrammetry for mapping and 3d modeling–current status and future perspectives. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 38(1), C22. 11. Tweaktown, (4/19/2015), http://www.tweaktown.com/news/42572/faa-issues-drone-permits-real- estate- agriculture-commercial-use/index.html Conclusions There is a wide array of imaging and computer vision applications for UAVs with the ability to impact many industries such public safety, agriculture, and engineering. One such application is post-disaster reconnaissance and infrastructure assessment. Before images and video can be utilized, post- processing for lens corrections is required. A MatLab program has been written for automatically correcting radial distortion in photos and videos taken by a Phantom 2 Vision+ UAV. The code is flexible enough to be adapted for other cameras and UAVs. The lens correction has been integrated with a simple crack detection algorithm and will be incorporated with detection of other features related to geotechnical engineering. Abstract Immediately following natural disasters, such as earthquakes, reconnaissance studies are performed to collect data and observe damage to infrastructure and geotechnical systems. However, access to sites is often limited due to safety considerations, difficulty and time. An Unmanned Autonomous Aerial Vehicle (UAAV) capable of gaining access to these areas could solve many problems and lead to more efficient post-disaster reconnaissance. A UAAV site reconnaissance and characterization platform is being developed. Among the many data collection features, the UAAV will collect photos and videos used to identify damage and features of interest. Commercial Unmanned Aerial Vehicles (UAVs), for performing preliminary field testing, were compared. The DJI Phantom 2 Vision + was selected after an investigation of the UAVs most commonly used with image processing techniques. Photos and videos recorded by the Phantom are used as the basis for calibrating image processing methods for identifying geotechnical features of interest. Before these aerial images and videos can be used for this purpose, significant post- processing is required. A MatLab program was developed to automatically detect photos and videos and batch process them to apply the necessary corrections. A correction for lens distortion is applied to remove the barrel effect (also known as fisheye effect) caused by the Phantom camera lens. Once the photos are corrected, they were used for automated crack detection. Acknowledgements The authors would like to acknowledge funding from Rackham Graduate School of the University of Michigan – Ann Arbor through a Rackham Graduate Student Research Grant. The authors would like to thank Bob Spence and Jan Pantolin for efforts in constructing the indoor flight facility. The graduate student is further funded by NSF grant award #1362975. Image Processing Lens Correction • Correct radial lens distortion (“fisheye”) • Inputs: Intrinsic Matrix and radial distortion coefficients specific to the camera Crack detection • Performed on grayscale images (ignoring color) • 2D Gaussian filter is applied to the image • The low-pass filter smooths the image • Possible cracks are traced by detecting gradients above a specified threshold Applications of Drone Technology Agriculture • Health monitoring, crop duster,… Huang et al. (2013) Atmospheric Measures • Pollution, meteorology,… Cinematography and Photography • Aerial views of events, movies,… Delivery • Home delivery, medical supply delivery,… Disaster Assessment • Earthquakes, tornado, floods, wildfire, search and rescue,… Baiocchi et al. (2013) Mapping • Remote mapping, 3D mapping, archeology… Remondino et al. (2011) Disaster Reconnaissance Objective Development of a UAAV site reconnaissance and characterization platform: • Photo and video recording; • LiDAR scans to collect point clouds of surface deformations; • Wireless sensor deployment of set predetermined geophone arrays, • Perform in situ shear wave velocity measurements. Method The research focused first on understanding the state-of-the-art practices and applications of imaging drones with a literature review. Once the DJI Phantom 2 Vision+ was chosen, the images/videos acquisition was conducted by operating the drone in an indoor cage. MatLab codes for post-processing the collected images were developed & incorporate feature detection algorithms. (11) (1) (4) (7) (5) (2) (9) DJI Phantom 2 Vision + Specifications Aircraft Specifications Battery Autonomy 20-25 min Communication Distance (open area) 500-700m Hover Accuracy (Ready to Fly) Vertical: 0.8m; Horizontal: 2.5m Onscreen Real-Time Flight Parameters Photography and Waypoints 3-axis High Performance Gimbal Control Accuracy: ±0.03° FC 200 Camera Operating Environment Temperature 0℃-40℃ Sensor size 1/2.3" Effective Pixels 14 Megapixels Resolution 4384×3288 HD Recording 1080p30 & 720p Recording Field Of View (FOV) 110° / 85° (10) (10) Lens Correction Problem: “Fisheye” lens causes barrel distortion Batch Process User input: Save video frames or not and at what interval? Correct images Save frames from video Save images & EXIF data Start End Input Images or videos Correct frame by frame and saves in a new corrected video Collect photos and images Save frames or not Uncorrected Corrected (10) (10) (11) (11)