Jan Mitrovics, JLM Innovation GmbH From Research to Real Life Experiences and practical demonstrations
Jan Mitrovics, JLM Innovation GmbH
From Research to Real LifeExperiences and practical demonstrations
• About JLM Innovation
• From research to real life
• Some history on electronic noses…
• How to develop a new sensor system?
• Application economics!
• Example 1 Instrumentation
• Example 2 Mass Market
• Exercises with chemical sensor systems
• Brief introduction to signal processing / data analysis
• MOXstick – Modulated single sensor
• Display of different systems
Structure of the talk
Jan Mitrovics, JLM Innovation GmbH 2
Founded in 2004 to help bridging the gap between R&D
and commercial exploitation.
20 years of experience in multi sensor systems in a broad
range of applications.
Products:
• Hardware platforms for various sensor technologies, interfaces and
networks
• Software platforms for data processing and control
Services:
• Custom hardware & software development based on a portfolio of
internal technology platforms
• Application support, feasibility studies, technology consulting,
market studies
About JLM Innovation GmbH and myself
3Jan Mitrovics, JLM Innovation GmbH
Efficient, easy to use solutions!
Added value, through access to
complementary technology.
Fair and open relationship.
Tailored solutions that meet your needs.
Prototype or small scale production.
Flexible licensing.
What to expect from us
4Jan Mitrovics, JLM Innovation GmbH
Sensor systems and sensor networks based
on a broad range of sensor technologies
Hardware platforms
QMB, SAW,
MOS, EC, CP,
FET, IR, ...
5Jan Mitrovics, JLM Innovation GmbH 5
Data acquisition and analysis software for
sensor systems
• Software
(Windows, Mac,
IOS, Android)
• Embedded Software
Software platforms
7Jan Mitrovics, JLM Innovation GmbH 7
€ 5M Nexus Report (in 2002)
€ 10-15M Wall Street Journal (1998)
€ 10M Greenberg (1998)
€ 12M Attempto Service GmbH (in 2001)
€ 50 M FutureTech Report
€ 145 M Gardner / Bartlett (in 2000)
€ 1200 M German Infotech (in 2004)
€ 4500 M The Economist (1998)• Source: Gardner 1st Workshop NOSE II
Estimates of Enose Market
9Jan Mitrovics, JLM Innovation GmbH
9 Commercial E-nose Companies in 1997
10
Source: Gardner and Bartlett, Electronic Noses 1999, OUP
Jan Mitrovics, JLM Innovation GmbH 10
Step 1: build or buy an enose
Step 2: measure some stuff
Step 3: do magical data analysis
Step 4: write publications
Step 5: go to Step 2
Typical early enose research
Jan Mitrovics, JLM Innovation GmbH 11
17 Commercial E-nose Companies in 2002
12
Source: Vanneste, Handbook of Machine Olfaction (2002), Wiley-VCH
Jan Mitrovics, JLM Innovation GmbH 12
23 Commercial E-nose Companies in 2005*
13
Source: JLM Innovation / NOSE II, Market Survey Electronic Nose 2005
* With products on the market
Manufacturer Products TechnologiesAirsense GDA, PEN, i-PEN/MOD, KegControl,
Proc.Ctrl Nose, EDUMOS sensors, IMS, thermo desorption
Alpha M.O.S. Fox, Prometheus, Gemini, Kronos, Astree MOS, CP and QMB sensors, MS, GC, Electronic Tongue
AltraSens OdourVector QMBAppliedSensor Embedded modules MOS, FE and QMB sensorsDr. Födisch Umweltmesstechnik AG OMD 1.10; OMD 98 MOSElement FreshSense MOSEnvironics Oy Chempro 100, ... IMS, sensor arraysElectronic Sensor Technology Znose GC with SAW detectorFive Technologies GmbH QMB 6, MS-Sensor, SensiTOF QMB sensors, MSGERSTEL GmbH & Co. KG ChemSensor MS, GC, MOS, TDSIllumina Inc BeadStation Bead arraysLennartz electronic GmbH MOSES II MOS and QMB sensorsMeridiantek AG / Sensobi DL 1000, DL 1000 IS, DL 1000 IS Smoke SCP sensorsMicrosensor Systems Inc VaporLab, Hazmat, Eagle Monitor ... SAW sensors, GCPerkin Elmer QMB6 QMB sensorsQuartz Technology Limited QTS-1 QMB sensorsRST Rostock System-Technik GmbH SAM detect, ... MOS sensorsSACMI EOS 835 MOS sensorsScensive Technologies Ltd Bloodhound, ST214 CP and DLC sensorsSMart Nose Ltd SMart Nose MSSmith Detection /Cyrano Sciences Cyranose 320 Carbon black polymer sensorsSysca AG Kamina MOSTechnobiochip Libra Nose QMB sensors
Jan Mitrovics, JLM Innovation GmbH 13
• Quality assessment of food and beverages
very fragmented, low volume, various deployments
• Pharmaceutical and chemistry applications
very fragmented, low volume, various deployments
• Medical applications
Highly regulated, mid to high volumes, no deployments yet
• Safety and military applications
Mid volumes, various deployments
• Environmental and agricultural applications
Fragmented, low to mid volumes, few deployments
• Embedded Applications
Mid to high volumes, very cost sensitive, few deployments
Application Areas and Potential Markets
14Jan Mitrovics, JLM Innovation GmbH 14
• Generic instruments, handheld instruments, dedicated systems and
embedded devices were covered
• More than 30 companies involved in electronic nose instrumentation
and technology
• Applications in food industry, chemistry and pharmaceutics,
medicine, safety and military, environmental and agricultural, and
automotive
• Market for generic instruments is very fragmented with low growth.
• Dedicated systems for medical and safety applications have a lot of
potential. Several companies are focused on these markets.
• Simple embedded devices are being introduces to automotive and
consumer markets with strong growth.
• For many application further improvement of the technology is
necessary.
Electronic NOSE survey 2005
15Jan Mitrovics, JLM Innovation GmbH 15
A „We use a system, with sensing properties that
are vaguely known, measuring something of
vaguely known composition and use magical
data analysis to generate the result that we
want.“
Application approach: Which one is better?
B „We know (or research) the compounds of
interest and then build a sensor system specific
for those compounds.“
Jan Mitrovics, JLM Innovation GmbH 16
„A sounds great. We can start right away
and get going. Why go through more
hazzle, when we can find out with some
simple tests.“
Optimists answer
Jan Mitrovics, JLM Innovation GmbH 17
A is pure gambling. Our chance to succeed
is nil. I need more infomation, so please
go back and spend more research until we
have enough information to do B.
Pessimists answer
Jan Mitrovics, JLM Innovation GmbH 18
Obviously B is the only way to go.
We need verifiable well founded
methodology and B is the only way to
achieve that.
Please increase our budget. (We need more
fundamental research and fancier
equipment for a proper assessment.)
Scientific answer:
Jan Mitrovics, JLM Innovation GmbH 19
We just sold 1000 instruments to our
promising key customer.
Now get us the damn instruments.
By the way, if you could make them for half
the price, we could sell 10000
By the way, if you could make them for less
than 10 bucks, we could sell a million of
em (and it wouldn‘t matter if they worked).
Marketing answer
Jan Mitrovics, JLM Innovation GmbH 20
Marketing has identified a great window of
opportunity.
I know you guys are cookin something. I
don‘t understand the details, but we need
something that works.
Please put together a project plan with
budgets and a timeframe. We have a
board meeting on monday and need to
take a decision.
Management answer
Jan Mitrovics, JLM Innovation GmbH 21
It depends!
• On the amount of preexisting knowledge
(or the cost of acquiring it).
• On the available technologies.
• On the difficulty of the application.
A and B are not mutually exclusive. Often
both approaches are followed
simultaneously, or repetitively!
The only true, correct answer:
Jan Mitrovics, JLM Innovation GmbH 22
Odor (human perception) > generic
Complex mixture > simple mixture
Variable samples > stable samples
Variable matrix > stable matrix
Minor compound > major compound
Trace detection > high concentration
Unknown analytes > known composition
Reference samples hard to get > easily available
High thru put / short measurement time > no time constraint
Traceability required > not required
No false predictions > indication
Certifications > unregulated
Criteria for application difficulty
23Jan Mitrovics, JLM Innovation GmbH 23
Typical error sources
24
Sample Measurement Evaluation
Pre treatment /Sampling
Sensors ElectronicsSpot sample Transportation /storage
Non represantative spot sample,
changing matrix,Influence of temperature, pressure,
humiditz, contaminations during sampling, transport and storage.
Contamination,
Influence of temperature,
pressure, carrier gas, amount and volume
on headspace
Drift,
poisoning, temperature,
pressure, carrier gas,
reference gas, gas flow.
Component
drift,temperature,
noise.Resolution or
measurement range too
small
Error in reference data
or recalibration data.Numerical effects.
Inadequate algorithms.
Jan Mitrovics, JLM Innovation GmbH 24
Economical potential
• Volume (number of deployments)
• Cost sensitivity / gross margin
• After sales
Development effort
Development risk
Initial costs for production
Sales cycle
Spin off opportunities
Application economics
25Jan Mitrovics, JLM Innovation GmbH 25
Application Steps
26
Problem definition
Feasibility prephase
Training
and Validation
Deployment
Specifications / Requirements
Expert analysis
Tests with limited sample set
Extensive sample set (all classes,
good bad, cross sensitivities)
Reference methods
Expert know-how
Installation at (multiple) sites
Jan Mitrovics, JLM Innovation GmbH 26
Feasibility Study
-> Check for discrimination (e.g. PCA-Plot)
-> Assess main components / cross interference
=> No final answer for success!
Application Development
-> Optimization of method (technology, ...)
-> Validation
-> Reproducibility, Reliability, ...
=> Ready for implementation (or not...)
Feasibility vs. Application dev.
27Jan Mitrovics, JLM Innovation GmbH 27
• First contact
• Second contact: Initiation of collaboration
Feasibility study
• Discussion + budget allocation
• Application development
• Acceptance + budget allocation
=> Repeat sales
Time to successful application
28
Months
0
2-3
6 – 12
> 6
6 – 12
======
Min 2
years
Jan Mitrovics, JLM Innovation GmbH 28
Classical Enose: Cost per sale
29
Instrument development
Instrument manufacturing
Marketing
Feasibility
Application development
Customer support
x M€ / sold instruments
x k€
Marketing cost / sold instr.
0.x k€
0.x M€ / (sold instr./ appl.)
x k€
Optimum: Few applications with high numbers
Real world:Average sale per customer: 1.x
Jan Mitrovics, JLM Innovation GmbH 29
Price Cost Manufacturing cost
-> Price = Customer Value
Be efficient
-> Don‘t waste your money on
-> impossible applications (technology fit)
-> non profit applications (nice to have)
Adopt your business modell
-> Concentrate on your strength
-> We need more specialization and open
exchange
Success strategies ?
30Jan Mitrovics, JLM Innovation GmbH 30
Example 1:
Quality control of packaging material
From research to real life
31Jan Mitrovics, JLM Innovation GmbH 31
Examples: PARFUM / ESCAPE
32
Philips-DAP, Netherlands
Philips-LEP, France
CSEM, Switzerland
Microsens, Switzerland
MOTECH, Germany
Neotronic Scientific, UK
Nestlé, Switzerland
AppliedSensor, Sweden
Gerstel, Germany
INRA, Dijon, France
Wall, Austria
Univ. Tübingen, Germany
M O T ECH
Jan Mitrovics, JLM Innovation GmbH 32
Quality control of packaging material
33
State of the Art:
Human Odour Panel
Automated investigation
with “Electronic Nose”
Better called application specific
sensor system (a triple s)
Jan Mitrovics, JLM Innovation GmbH 33
Scoring
34
Evaluation of odor and taste in comparison with a reference material. The difference is estimated according to the following category scale:
• 0: no difference with the reference
• 1: hardly perceptible difference (not definable)
• 2: slight difference (just definable)
• 3: strong difference (clearly definable)
• 4: very strong difference
Any packaging materials with a median score equal or greater than 2.5 is considered to be of doubtful quality and should be submitted to further analyses.
Jan Mitrovics, JLM Innovation GmbH 34
Quality control of packaging material
35
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Od
ou
r p
red
icti
on
by M
OS
ES
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Test data withre-calibration ofthe array
Training data
Odour prediction by human sensory panel
RMSE = 0.26 RMSE = 0.57
Od
ou
r p
red
icti
on
by M
OS
ES
II
Jan Mitrovics, JLM Innovation GmbH 35
GC/MS of Nestlé Packaging Material
36
5 10 15 20 25 30 35
0
1
2
3
4
5
Inte
ns
ity
[a
.u.]
Retention time [min]
more than 70% cyclohexane
in the headspace
Jan Mitrovics, JLM Innovation GmbH 36
Quality control: Final Product
38
Application specific sensor system (a triple s)
Jan Mitrovics, JLM Innovation GmbH 38
• Odor quality assessment by panels delivers imprecise
data
• Solutions are possible in simple cases (few components)
• Reference analysis is critical
• Methods cannot easily be transferred to new sample
types
• Reference sample sets are expensive and hard to get
• Successful, validated method does not guaranty
commercial success
Quality Control: Lessons learned
39Jan Mitrovics, JLM Innovation GmbH 39
• 2 consecutive projects, total duration more than 6
years
• Commercialization after the second project
• Effort for commercialization low in comparison to
research projects
• Prototypes during project based on commercial
products
• Low volume
• Small certification requirements
• No challenging production cost constraints
Project characteristics
40Jan Mitrovics, JLM Innovation GmbH 40
Example 2:
Control of air intake in automobiles
From research to real life
41Jan Mitrovics, JLM Innovation GmbH 41
Examples: CIA
42
VDO Germany
FIAT, Torino, Italy
Telecom Italia
Univ. Warwick, UK
Univ. Southampton, UK
Univ. Rome, Italy
Univ. Linköping, Sweden
Univ. Neuchâtel, Switzerl.
Univ. Tübingen, Germany
Jan Mitrovics, JLM Innovation GmbH 42
Automotive application
43
Automatic switching of the re-circulation flap• Increase of cabin air quality
• Safety issue
• Comfort aspect
Jan Mitrovics, JLM Innovation GmbH 43
• Development of prototype instruments containing
different sensor technologies
• Real life tests with the prototypes (test drives)
• Optimization of the sensor technologies for the
application
• Development of pattern recognition techniques
Methodology during project
44Jan Mitrovics, JLM Innovation GmbH 44
• Duration 3 Years
• Outcome of the project
• Optimized prototype gas sensors
• Selected sensor technology
• Development methodology
• Effort for commercialization very high!
• Prototypes used in project not suitable for
commercialization (complete redevelopment)
• High certification requirements
• High volume, very cost sensitive
• New partners required
Project characteristics
46Jan Mitrovics, JLM Innovation GmbH 46
• Certification / Design Wins / Product cycles
• Upscaling of production
• Establishing of a production process
• Reproducibility / Stability
• No recalibration
• No / limited factory calibration
• Cost, Cost, Cost
• Total solution cost (sensor + packaging + electronics +
signal processing + calibration)
• Better than good enough is wasting money!
Specific challenges of mass markets
Jan Mitrovics, JLM Innovation GmbH 47
The SNet sensor network system:
4 Metal oxide sensors
32bit µC for onboard
data analysis
Ethernet and Zigbee
communication
Server software for
operation as sensor network
Display of chemical sensor systems
Jan Mitrovics, JLM Innovation GmbH 49
The MOXstick:
1 Metal oxide sensor, almost any available
MOx sensor can be used.
Temperature modulation and sensor
characterization
Plug and Play, no drivers required
Simple, but powerful PC software
Sensor arrays by combining
several Sticks / Instruments
Experiments with the MOXstick
Jan Mitrovics, JLM Innovation GmbH 50
Typical Steps During Data Analysis
Jan Mitrovics, JLM Innovation GmbH 51
final result
Feature
extractionmax, min, area, ...
Pattern
RecognitionPCA, LDA, ANN, ...
4
3
2
1
x
x
x
x
Pre-Processing: Filters
Jan Mitrovics, JLM Innovation GmbH 52
Filters are often applied to reduce noise in the signal
Clean signal
Noisy signal
MA3
MA30
IIR
Median5
xax x
ai
i i alt
,
1
xN
xi i jj
N
1
1,Moving Average
IIR
Pre-Processing: Linearization
Jan Mitrovics, JLM Innovation GmbH 53
Many simple pattern recognition algorithms rely on linear calibration curves.Non linearities in the transducer or electronics are often well known.→ Linearization during pre processing is a simple yet very effective method
Input e.g. concentration
Ou
tpu
t e
.g. V
olt
)(xfy
Input e.g. concentration
Lin
ea
rize
dO
utp
ut
)(1 yfconstylin
2
)ln(
yy
yy
yy
lin
lin
lin
Feature Extraction: Basic Features
The complete response curve contains a lot of redunancy.
In order to compute robust results, we want to remove as much
redundancy as possible, while preserving a maximum of information!
Jan Mitrovics, JLM Innovation GmbH 54
Se
nso
r sig
na
l
time
Maximum
MinimumBaseline
Signal at a
certain point
in time
Area
Pe
ak h
eig
ht
Feature Extraction
Criteria for good features
• High correlation to the desired information
• Low correlation to other features
• Robust
• Not susceptible to noise
• Reproducible (e.g. masks variations in flow,
temperature, memory effects …)
• Linear with changes in input signal
Jan Mitrovics, JLM Innovation GmbH 55
Common Features related to signal strength
Maximum:
• directly related to the size of the input value
Signal Height (Maximum – Baseline):
• removes offset and additive baseline drift from the sensor response
• typical use: QMB sensors, measurements with changing background
Relative Signal Height (Maximum – Baseline) / Baseline:
• weights the signal height with changes in baseline
• may be used to counteract linear drift effects, that affect baseline and sensitivity in
the same way
• scaling of different sensors to similar sensitivity
• typical use: metal oxide sensors sensor
These features are often combined with
averaging over a number of measurement points to reduce noise
ln(feature) or other linearisation functions
Jan Mitrovics, JLM Innovation GmbH 56
Jan Mitrovics, JLM Innovation GmbH 57
-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5
PC1: 66.5%
-4
-3
-2
-1
0
1
2
3
4
PC
2:
16
.4%
Anisol
Cyclohexanon
Propanol
Toluene
PCA: Basic concept
The PCA scores plot represents
a 2-dimensional view on the
data, in which
• the maximum of
variance is contained
• while preserving the
relative distances
between the objects.
• Often the amount of
information represented
by one principal component
is printed in the axis label.
• The principal components are
always sorted by their
information content, with PC1
beeing the most important one.
Jan Mitrovics, JLM Innovation GmbH 58
PCA: Calculations
In PCA X is decomposed (e.g. via SVD or NIPALS) to:
Where:
U is a matrix containing the Eigenvectors of XXT,
V is a matrix (called loadings) containing the Eigenvectors of XTX and
S is a diagonal matrix containing the square roots of the Eigenvalues λ.
Note that XTX corresponds to the covariance or correlation matrix of X
(depending on the scaling of the variables).
The scores for the objects in X are simply the Eigenvectors in U multiplied by S.
(The Eigenvectors in U and V are normalized to length one).
To calculate the scores t for a new object, we multiply its measurement vector x
with the loadings V :
Vxt
idiagS,USVXT
Jan Mitrovics, JLM Innovation GmbH 59
-1.2 -1.0 -0.8 -0.6 -0.4 -0.2 -0.0 0.2 0.4 0.6 0.8 1.0 1.2
PC1: 66.5%
-0.8
-0.6
-0.4
-0.2
-0.0
0.2
0.4
0.6
0.8
PC
2:
16
.4%
Q1
Q2Q3
Q4
Q5
Q6
Q7
Q8
S1
S2
S3 S4
S5
S6
S7
S8
PCA: Loadings plot
The loadings plot shows the
influence of each variable
onto the principal
components.
Variables not contributing will
be near the center.
Variables with similar
information content will be
represented next to each
other.
Jan Mitrovics, JLM Innovation GmbH 60
PCA: Biplot of scores and loadings
o In a biplot, scores and
loadings are shown (with
adequate scaling)
o By comparing the directions of
the loadings with the position
of objects it is possible to draw
conclusions on the suitability
of individual variables.
o In this plot the variables S2 –
S7 are in the direction of the
spread within the groups.
-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5
PC1: 66.5%
-4
-3
-2
-1
0
1
2
3
4
PC
2:
16
.4%
Anisol
Cyclohexanon
Propanol
Toluene
Q1
Q2 Q3
Q4
Q5
Q6
Q7
Q8
S1
S2S3S4
S5
S6
S7
S8
Jan Mitrovics, JLM Innovation GmbH 61
PCA: Scree plot
o The scree plot shows the
amount of information
represented by each principal
component (bars)
o It is also possible to show the
information content covered by
the first x components (line)
o The information content of a
principal component is the size of its Eigenvalue λ, scaled in a
way, that the sum of all
Eigenvalues is 1.
0,00%
10,00%
20,00%
30,00%
40,00%
50,00%
60,00%
70,00%
80,00%
90,00%
100,00%
PC 1 PC 2 PC 3 PC 4 PC 5 PC 6 PC 7 PC 8 PC 9 PC 10 PC 11 PC 12 PC 13 PC 14 PC 15 PC 16
The scree plot helps to identify the number of
components, necessary to explain the data. In
above plot, the first 3 components amount to
more than 90% of all information.