Feasibility Study: Digitalisation of Collaborative Human-Robot Workspaces Joanna Turner, John Hodgson, Istvan Biro, Ella-Mae Hubbard, Andrea Soltoggio, Peter Kinnell, Niels Lohse Intelligent Automation Centre Loughborough university 1. Introduction Collaborative human-robot workspaces will be essential to increase productivity and competitiveness in manufacturing. One of the most challenging barriers to employing these technologies is the need for real-time awareness of the workspace, to ensure the safety of all actors. At present, safety comes at the cost of productivity. This study investigates the use of open source state-of-the art machine learning computer vision tools in combination with a network of multiple standard 2D cameras and classic 3D reconstruction techniques to detect and localise people and objects in the 3D workspace. Through the application of different deep learning algorithms, including OpenPose for key point detection, and DeepLab for semantic segmentation, the potential for real-time digitisation of the human-robot workspace was assessed. Robust skeletal reconstruction 2. Vision AI applied to Manufacturing 4 State-of-the-art algorithms were evaluated. They were tested to determine their performance in a manufacturing environment. All the algorithms successfully detected people or objects. However, they also all made mistakes. OpenPose Mask R-CNN DensePose DeepLab 3. Hardware performance comparison Each algorithm was evaluated on a range of processors. • 20x increase in speed from CPU to GPU. • Algorithm efficiency and image size are critical for speed. 5. Conclusions and Outcomes This feasibility study investigated the possibility of real-time digitisation of 3D manufacturing environments from a set of 2D standard RGB cameras. The key findings were: • Advanced algorithms and hardware were evaluated in a manufacturing environment. • Current state-of-the-art algorithms do not recognise uncommon objects that might be important features in a specialised manufacturing environment. • Near real-time 3D tracking of humans, without markers, in an industrial environment is feasible. • GPUs are becoming faster over time. • In future, more complex and accurate algorithms will also run in real-time. 6. Future Work • Create dataset for industrial objects (expensive) • Train models to detect industrial objects • Deploy to embedded devices Frames per second 0.0 20.0 40.0 60.0 80.0 CPU K80 1070 ti 2080 ti Mask R-CNN (1024x1024) DensePose (512x512) DeepLab (512x512) OpenPose (512x512) 4. Data fusion for fast & robust 3D tracking Applying OpenPose on 8 RGB cameras and applying a triangulation step results in fast and robust tracking of people in 3D space.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Feasibility Study: Digitalisation of Collaborative Human-Robot Workspaces
Joanna Turner, John Hodgson, Istvan Biro, Ella-Mae Hubbard, Andrea Soltoggio, Peter Kinnell, Niels Lohse
Intelligent Automation CentreLoughborough university
1. IntroductionCollaborative human-robot workspaces will be essential to increase productivity and competitiveness in manufacturing. Oneof the most challenging barriers to employing these technologies is the need for real-time awareness of the workspace, toensure the safety of all actors. At present, safety comes at the cost of productivity. This study investigates the use of opensource state-of-the art machine learning computer vision tools in combination with a network of multiple standard 2Dcameras and classic 3D reconstruction techniques to detect and localise people and objects in the 3D workspace. Through theapplication of different deep learning algorithms, including OpenPose for key point detection, and DeepLab for semanticsegmentation, the potential for real-time digitisation of the human-robot workspace was assessed.
Robust skeletal reconstruction
2. Vision AI applied to Manufacturing4 State-of-the-art algorithms were evaluated. They were tested to determine their performance in a manufacturing environment. All the algorithms successfully detected people or objects. However, they also all made mistakes.
OpenPose
Mask R-CNN DensePose DeepLab
3. Hardware performance comparisonEach algorithm was evaluated on a range of processors.• 20x increase in speed from CPU to GPU.• Algorithm efficiency and image size are critical for speed.
5. Conclusions and OutcomesThis feasibility study investigated the possibility of real-timedigitisation of 3D manufacturing environments from a set of2D standard RGB cameras. The key findings were:• Advanced algorithms and hardware were evaluated in a
manufacturing environment.• Current state-of-the-art algorithms do not recognise
uncommon objects that might be important features in a specialised manufacturing environment.
• Near real-time 3D tracking of humans, without markers, in an industrial environment is feasible.
• GPUs are becoming faster over time.• In future, more complex and accurate algorithms will
also run in real-time.
6. Future Work• Create dataset for industrial objects (expensive)• Train models to detect industrial objects• Deploy to embedded devices
Fram
es p
er s
econ
d
0.0
20.0
40.0
60.0
80.0
CPU K80 1070 ti 2080 ti
Mask R-CNN (1024x1024) DensePose (512x512)DeepLab (512x512) OpenPose (512x512)
4. Data fusion for fast & robust 3D trackingApplying OpenPose on 8 RGB cameras and applying a triangulation step results in fast and robust tracking of people in 3D space.
Related Documents
