SEWER SCOUTTM – FLOATING DRONE 3D CONDITION ASSESSMENT … · The first version of the Sewer ScoutTM floating drone was successful in inspecting an asset that had not been inspected

SEWER SCOUTTM – FLOATING DRONE 3D CONDITION ASSESSMENT OF SEWERS

Steve Barclay 1 1. Sydney Water Corporation, Sydney, NSW, Australia

KEYWORDS Sewer ScoutTM, Condition Assessment, Trunk Sewer, Innovation, Drone, 3D Photogrammetry, modelling, machine learning, safety, spacial location, point cloud, IoT, cloud. INTRODUCTION Sewer ScoutTM is an innovative floating drone that has the potential to change the way water utilities around the world condition assess and prioritise their large wastewater and stormwater assets in the future. Using the latest point cloud 3D photogrammetry technology and machine learning, it has the potential to automate the inspection and condition assessment process to minimise the safety risk of entering these confined space assets and provide more holistic quantitative data to improve the capital and operational expenditure to ensure the best value-added service to their customers.

Figure 1 Sewer ScoutTM Inspection 3D Drone YEAR CASE STUDY WAS IMPLEMENTED 2015 to 2018. CASE STUDY SUMMARY To demonstrate the suitability of the innovative 3D photogrammetry technology in live traversable sewers and stormwater conduits and then transfer that technology to a remote floating drone to eliminate the need for person entry traverse conduit condition assessment inspections. The final phase will be to automate the backend processing to analyse the gathered data and produce suitable engineering reporting outcomes to the WSAA code to enable the optimal operational and capital asset management investment decisions. CASE STUDY DETAIL • Limiting Person Entry Traverse Inspections. Sydney Water condition assesses about 300km of Avoid Fail wastewater and stormwater conduits every year. 70km of that is by man entry traverse inspection ranging in sizes above 1500mm in height. This process requires managing 13 of Sydney Waters’ 16 Fatal Safety Risk Standards and involves complex and robust

safety procedures to ensure the safety of the inspection party entering the confined spaces to undertake the condition assessment. As such, it is considered one of the riskiest activities that Sydney Water undertake and that innovative technology should be investigated to replace this method of condition assessment.

Figure 2 Steve Barclay of Sydney Water and Shariff Shockair of SASTTI sewer traversing the NSOOS. Conduit sizes ranging between 900-1500mm have already been moved from the traverse program to the CCTV program as the CCTV technology became available and was price competitive. CCTV technology in larger walk through conduits is limited by cable lengths, data transfer between operator and unit, lighting and operator experience. Currently it is more cost effective to inspect by person entry than CCTV units. • GeoInteractive 3D Photogrammetry Technology. Geointeractive is an iAccelerate start up company out of the University of Wollongong. They use the latest 3D photogrammertric technology to allow the construction of geolocated 3D models of structures such as sewers. This is done by using a series of photos which can be veiwed through any standard web browser removing the need for specialist point cloud software. Current standard inspection technologies can capture 360 degree images from a single point location. (Think of real estate 3D models of properties where you can scan around a room to see what it is like inside the propoerty without the need to visit it.) This is achieved by setting up a camera in the middle of the room and scanning the entire room and then putting the images into software to generate the 360 virtual tours. GeoInteractive have advanced this technology to be able to develop 3D models from a sequence of images. These images are then geolocated and constructed as 3D models of the sewer. The Geointeractive software behind the imagery is being developed to include machine learning and automatic recognition of standard sewer observationas as per the WSAA code to produce standard condition reports.

Figure 3 Robert Lee of GeoInteractive traversing the Western Branch Main collecting the 3D images.

• Summary of activities and their implementation. Stage 1 The first stage was to prove the technology in the sewer and design a reporting and viewing interface to determine if the technology was applicable to Sydney Water. This was achieved in late 2015 after a manned entry traverse inspection of the Western Branch Main.

Figure 4 Sewer ScoutTM Viewing Platform with the 3 data screens

The viewing interface combines the 3 views of location, CCTV view and the 3D model view. (See Figure 4 ) The first screen, Map View, shows the geolocated assets inspected in a Google ViewTM window. The Google Maps Pin can be moved and the coresponding Image View will move automatically to that location showing the condition view of the asset. The Image view can also be moved by scrowling the mouse for a simple fly through view of the asset. The third view, 3D View, shows the 3D model with points of interest taged for easy viewing. The 3D view can also open in a separate window to allow rotation and closer inspection of the asset. A second traverse in the Blackwattle Bay stormwater system further refined the viewing platform and the data analytics to produce work required maps as show below. This negated the need to undertake a separate Level 3 inspection to define the scope of required rehabilitation works, saving time and money in the rehabilitation development phase. See Figure 5.

Figure 5 Data analytics and geolocation of defects These two initial inspections proved the technology as suitable for use in both wastwater and stormwater conduit condition assessment and refined the display output and data analysis capabilities of the system. It also enabled better data collection and analysis processes. Both inspections used the person entry version in the Western Branch Main and then in the Blackwattle Bay stormwater system. Other inspections using the person entry version include the North Georges River Submain (NGRS) and Northern Suburbs Ocean Outfall Sewer (NSOOS 6-7) Figure 6 shows the viewing panel of the large Eve St merging chamber and 3D model. The 3D model gives a great overview of the size and scale of the structure which is not able to be conveyed using the standard traversing method. The model can be rotated and zoomed in or out to view details or overviews.

Figure 6 Viewing panel of the Eve St Merging Chamber.

Stage 2 The third inspection was to attempt the technology using a remote floating drone (See Figure 7) in the West Middle Harbour Sewer 1200mm diameter Section C, which had not been inspected since the 1980s due to its difficult access in a National Park and up to 115m deep intermediary maintenance holes. The Sewer ScoutTM was released and retrieved downstream without any complications. It took approximately 20 minutes to complete the 1445m survey. Whilst this trial was successful in inspecting the entire length and gathering enough data to make an engineering condition assessment, the data was not sufficient to be able to be used in the 3D model or a complete normal image view. The data collected did not give a complete 360 degree view of the entire conduit from water level to water level. See Figure 8. The data captured used a fish eye lens facing directly upwards to the obvert of the conduit.

Figure 7 Steve Barclay of Sydney Water and Robert Lee of GoeInteractive P/L Inventor of Sewer ScoutTM

Figure 8 View of the first Sewer ScoutTM drone using a single camera version.

Further development to re-design the Sewer ScoutTM with a multi camera head, as shown in Figure 1, to gather the required data from a live trial were successful in March 2018. A trial on the NGRS of 3178m of 2515mm pipe and box section, included two 90-degree bends, successfully gathered the data to produced full 3D models of the sewer with full 360 degree vision and tilt and pan capability. See Figure 9.

Figure 9 Viewing platform of NGRS trial multi camera Sewer ScoutTM drone with 3D model. Stage 3 To improve the data analytics and software to automatically generate condition assessment reports to WSA 05-2013 Conduit Inspection Reporting Code of Australia and Sydney Waters Years to End of Service Life (YESL) gradings. Initially, this was a manual process and has already progressed to have standard observations and gradings included in the viewing panel and geolocated in the 3D models. See Figure 11. Longer term, this will involve the use of machine learning technology to automatically identify standard structural and serviceability observations and grade them to the WSAA code and Sydney Waters’ YESL scores. See Figure 10. Work is already well advanced in the machine learning capability to achieve this automatic recognition and recording of both structural and serviceability grade observations. Work is continuing to refine this capability.

Figure 10 Machine Learning automatic observation recognition software.

A key feature of the Geointeractive software is it will allow future inspections to be overlayed and compared to show differences in the asset condition. This is critically important to the future of the asset management of these major infrastructure assets as they age and reach the end of their service life. Being able to make investment decisions based on actual quantitative data will be very important in achieving the optimal life cycle cost of the assets and the Sewer Scout software will help achieve this.

Figure 11 View of the YESL panel. The final task will be to integrate this work into Sydney Water’s IT and database platforms to ensure completeness of data capture and storage for future analysis. • Summary of outcomes and measurable impacts of the activities. So far, we have identified a technology that could possibly replace the majority of the current person entry traverse condition assessment inspections. This has the potential to reduce the safety risk profile of this work. The software viewing window has been designed and displays the data in a three-window panel showing its location, image view and 3D view. It is easy to use and navigate through the secure log in cloud based platform. It has successfully been trialled using the person entry technique and has now been successfully transferred to the non-person entry floating 3D drone stage. The first version of the Sewer ScoutTM floating drone was successful in inspecting an asset that had not been inspected for over 30years due to access and safety issues. Though it did not give a full 360-degree view or 3D model, valuable lessons were learnt to assist in redesigning the floating drone. It has now been redesigned to ensure a full Image View and 3D capability. The data analytics has already been used to produce detailed rehabilitation diagrams for the Blackwattle Bay stormwater system from the inspection, which saved time and money in the rehabilitation design phase by eliminating the need to undertake further detailed inspections. The machine learning component has advanced to automatically identify and log some basic observations. Work is continuing to improve this capability. Initial cost estimates are that the Sewer Scout is about half the cost of traditional traverse inspection techniques and is considerably faster at 1 metre per second (3.6km/hr) to gather the data as opposed to 1 km/hr for current traversing. Geointeractive have also successfully trialled converting a standard CCTV inspection into the software to determine if it could develop a 3D model. It could generate a Map View and Image View but is unable to

generate a 3D model view. See Figure 12 below. Further work would be required if 3D models are required to be developed from existing CCTV footage.

Figure 12 Converted CCTV footage.