3D Light-field Technology in Combination with Deep-Learning for Industry Philipp Schmid – Head Robotics & Automation
Mar 23, 2021
3D Light-field Technology in Combination with Deep-Learning for Industry
Philipp Schmid – Head Robotics & Automation
Copyright 2019 CSEM | Page 1
Our mission
Development and transfer of microtechnologies to the industrial sector – in Switzerland, as a priority – in order to
reinforce its competitive advantage.
Cooperation agreements with established companies
Encouraging the creation of start-ups
Copyright 2018 CSEM | Big Data – Smart Data – Fachgruppe Smart Systems; 12. Oktober 2018 | Alexander Steinecker| Page 2
CSEM – A TECHNOLOGY ORGANIZATION IN SWITZERLAND
The CSEM is a Swiss RTO with over 450 emps. Thecompany develops advanced microtechnologiesfor the industry. Thus it creates innovations.Focus areas are Indusry 4.0, e-health, advancedmanufacturing, clean energy.CSEM’s head quarters are located in Neuchâtel,Further branches are in Muttenz, Landquart,Alpnach and Zürich. CSEM is connected toInternational RTO’s such as FhG, VTT, CEA.In the past 10 years CSEM created over 40 Startup companies active in numerous markets.
Copyright 2018 CSEM | Big Data – Smart Data – Fachgruppe Smart Systems; 12. Oktober 2018 | Alexander Steinecker| Page 3
CSEM at a glance
437Persons
43New ventures
175Industrialclients
203Patent families
64Europeanprojects
43Nationalities
83.0Turnover(mio CHF)
Copyright 2019 CSEM | Page 4
CSEM – Swiss Research for Industry
N N A L M Z
N A L
MZ
N
CSEM Zürich
CSEM Muttenz
CSEM Neuchâtel
CSEM Alpnach
CSEM Landquart
Copyright 2019 CSEM | Page 5
Technology platforms to foster innovation
Advanced micro-manufacturing
Printableelectronics
Bio-surface engineering
MEMS
Nano-surface engineering
Scientific instrumentation
Medical technologies
Automation
Energy systems
Emerging & thin-film PV
PV cells& Modules
Vision systems
System-on-chip
Wireless
Industry 4.0
IoT
e-health
e-energy
Advanced manufacturing
Functionalpackaging
Copyright 2019 CSEM | Page 6
Robotics & Automation
Robotics
Vision
Control
Deep Learning
Copyright 2019 CSEM | Page 8
History
1908Gabriel Lippmann
Copyright 2019 CSEM | Page 9
100 years later…
2008Gabriel Lippmann
Market entrance 2010
Light field cameras for industry and research
Patented MLA design for optimal depth-of-field
Copyright 2019 CSEM | Page 10
Principle
Standard Camera Lightfield Camera
Copyright 2019 CSEM | Page 11
Lightfield (Plenoptic) Camera – 3D Vision
GPU Power
Copyright 2019 CSEM | Page 12
3D Lightfield camera
Copyright 2019 CSEM | Page 13
3D Lightfield camera
Copyright 2019 CSEM | Page 14
Extended depth-of-field
Lens Type 1 Lens Type 2 Lens Type 3
Copyright 2019 CSEM | Page 15
Multi Focus
Image plane Microlens plane
Lens Type 1
Lens Type 2
Lens Type 3
Copyright 2019 CSEM | Page 16
Extended Depth-of-field
Raytrix Lightfield Camera R5μStandard 4MP Camera
Copyright 2019 CSEM | Page 17
Occlusion
Stereo Camera Lightfield Camera
Copyright 2019 CSEM | Page 18
3D in production
Total Focus Image
3D imageRoundness
Statistics
Copyright 2019 CSEM | Page 19
Difficult Parts
Copyright 2019 CSEM | Page 20
Screws
Copyright 2019 CSEM | Page 21
BinPicking for small parts (under development)
• 3D Lightfield Camera
• CSEM Libraries
• Fast and Robust
Copyright 2019 CSEM | Page 22
Dental
Copyright 2019 CSEM | Page 23
3D Microscopy
FoV ~ 1x1mm Total Focus 3D
Copyright 2019 CSEM | Page 24
Solder Bumps
Copyright 2019 CSEM | Page 25
Product range
Copyright 2019 CSEM | Page 26
Quality Control Today
Copyright 2019 CSEM | Page 27
Traditional Image Processing…
Model
World
≠
Algorithm
Copyright 2019 CSEM | Page 28
Copyright 2019 CSEM | Page 29
How does our Brain work?
DEEP LEARNINGAT CSEM
10+ years experience
Support humans
Industry proven
Copyright 2019 CSEM | Page 31
Deep Learning since 2008 => Industrially validated solutions
CSEM
technology
transfer
Copyright 2019 CSEM | Page 32
Copyright 2019 CSEM | Page 33
Missing Link
Copyright 2019 CSEM | Page 34
Integrated Robotic Software Framework «VISARD»
PC-based, one stop solution
Flexible, extendable,
reusable and user-friendly
Database logging and
automatic management
report generation
Hardware independent
Online process
development with scripting
tool (similar IEC 61131)
Benefits for the users
Copyright 2019 CSEM | Page 35
Resolution
Parameters Overview system High-resolution system
Working distance 48 mm 20 mm
Field of view 7.8 mm x 7.8 mm 0.65 mm x 0.44 mm
Lateral resolution 3 m 0.7 m
Depth of field 1.1 mm 0.06 mm
Depth resolution 2.5 m 0.4 m
Framerate (max) 81 fps 30 fps
www.citcom.eu
Funded by the European Union’s HORIZON 2020 programme, Grant Agreement no. 768883
Copyright 2019 CSEM | Page 36
3D Vision Demo
Copyright 2019 CSEM | Page 37
CSEM Vision Benefits
• CSEM source code – no third party libraries (Cognex, Halcon, Matrox)
• No third party license fees
• No black box, not depended on external support
• CSEM has control over every bit and byte -> perfect adjusted solution
• Use of latest machine learning technics
• Modular software in C# .NET (supporting many camera brands)
• Easy to customize
• Full data logging (folder structured or database)
• Connection to PLC (ADS, OPC, …), TCP/IP, RS232, …
Contact:
Philipp Schmid
Head Robotics & Automation
T +41 41 672 75 62
M +41 78 769 86 78
CSEM SA
Untere Gründlistr. 1 | 6055 Alpnach Dorf
www.csem.ch