"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection

Page 1

The Reverse Factory

Embedded Vision in High-Volume

(and Value) Laboratory Applications

Patrick [email protected]

Embedded Vision Alliance

Hamburg 6th September 2017

V2.6

Page 2

Information-rich image sets in the hands of users

Page 3

Fred Sanger (1918-2013)

• Nobel Prize 1958

• Protein sequencing

• Human insulin

Image credit: MRC Laboratory of Molecular Biology

Page 4

Structure of DNA 1953

Crick and Watson

Nobel 1962

Friedrich Miescher 1869

Page 5

Fred Sanger (1918-2013)

• Nobel Prize #2

• DNA sequencing 1980

1st generation sequencing

C T G A

Image credit: MRC Laboratory of Molecular Biology

separa

tion b

y e

lectr

ic fie

ld

Page 6

Synopsis

• Motivation: the need and the market

• Laboratory as a factory in reverse

• Enabled by science and technology (including imaging)

• Big applications today: NGS case study

• End applications: ourselves and our world, family, food

• How it works: chemistry, optics, software

• Role of imaging in delivering performance

• Improvement curve: Carlson’s curve and what this means

• Cost, speed, growing the market, new applications

• The next applications for imaging

• Scientific & technological trends

• There are still plenty of opportunities

Page 7

Laboratory as a factory in reverse:

from sample to information

petrochemicals

Industrial

biomedical

research

pharmaceutical

forensicsenvironmental

materials

research

food & drink

consumer

goods

from well behaved to heterogenous; from solid into liquid form

clinical

Life sciences Physical sciences

Page 8

petrochemicals

Industrial

biomedical

research

pharmaceutical

forensics

materials

research

food & drink

consumer

goods

clinical

Life sciences Physical sciences

Pharma

R&D

$50bn

Clinical

Testing

$60bn

Lab

instruments

$40bn

forensic

testing

$20bn

food

testing

$11bn

environmental

Increasing use of imaging

Laboratory as a factory in reverse:

from sample to information

Page 9

Clinical applications of genomics

• Screening

• Diagnosis

• for cancer, infection

• Treatment

• for selection, progress, follow up

• example: breast cancer BRCA1

• Emerging area

• counselling and reproduction

cisncancer pharmainfo.net

Page 10

Applications expanding beyond medicine

• Next generation sequencing is now used very widely

• Family

• Food

• Flu

• Forensics

• Fish

• High volume applications of NGS: all that touches on life

Page 11

Family: self, ancestry, genealogy

• Self

• Inheritance

• Health risk?

• Regulation

• FDA and terms of use

• (and our pets)

consumer

goods

Page 12

Flu: Tracing infection Zika 2016

• 4-40 entry points from April

Grubaugh, Nathan D., et al. "Genomic epidemiology

reveals multiple introductions of Zika virus into the

United States." Nature (2017) 546, pp401-405.

Page 13

Aircraft safety: bird strike data - what when and why

Lapwing

Kestrel

Galah

environmental

Page 14

Elements of an NGS (next generation) system

• DNA strand

• Flow cell

• Chemistry

• Optics

• Laser

• Camera

• Software

https://www.youtube.com/watch?v=9YxExTSwgPM sequencing 5min

https://www.youtube.com/watch?v=pfZp5Vgsbw0 flow cell 2min

illumina

Page 15

Role of imaging: the flow cell

Each image 3-4Mp, 120k images per 36 cycle run = 350Gb

Page 16

Role of imaging: the optical path

illumina

fluorescence

Page 17

Role of imaging: how it works

8 lanes x 100 tiles. 70bp -> 28k images / lane

300k clusters per tile. 3Gb totalillumina

Page 18

Information-rich image sets in the hands of users

Page 19

Further improvements (1)

Problem: 4 colour channels per image

illumina

Page 20


Problem: 4 colour channels per image

Solution: from 4 channels to 2 channels

illumina

Page 21


Leads to other problems

• But …. 2 colour chemistry can overcall high confidence G bases

The sequence below shows this effect:

@1:11101:2930:2211 1:N:0

ATTTATTATTAATTAAATATTAATAATAAATAGATCGGAAGAGCACACGTCTGAACTCCAGTCACTAGCTTAGCGCGTATGCCGTCGTCGGCGTGCAAAAAAAAAGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG

+AAAAAEEEEEEEEEEEE6EEEEEAEEEEEEEEEA/EE<EEEAEE/EAEEAEEEE6</EEEEEA/<//<///A/A//////</E<//////E///A/</A/<<A////A/E<EEEEEEEAEEE/EEEAEAEAEAE6/AEAEE<AAEAEE

It’s easier to see if you visualise the quality scores for this sequence

single_seq_quality

https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/

Page 22


Problem: cluster density issues

• Cluster density can be demanding

• Especially for some samples

Krueger F, Andrews SR, Osborne CS (2011) Large Scale Loss of Data in Low-Diversity Illumina Sequencing Libraries Can Be Recovered by

Deferred Cluster Calling. PLOS ONE 6(1): e16607. https://doi.org/10.1371/journal.pone.0016607

Page 23


Solution: patterned flow cells

50nm spots spacing 250nmillumina

Page 24


Solution: patterned flow cells

• From random spots to fixed positions

• Simplification of analysis

• Increase in density and reliability

• So more data in less time, cost

illumina

Page 25


Problem: better use of flow cell

• Solution: use two surface imaging

• Challenging imaging and focussing

• End up with an optical head with 6 linear cameras

Illumina US8143599

Page 26

Nature 507, 294–295 (20 March 2014)

Improvement: Carlson’s curve and what this means

Page 27

Improvement: Carlson’s curve and what this means

• average of 5x in 2 year (2.4x faster than Moore)

• with a peak of 1000x in 2 years

• 100k x better over 14 years vs 34 years

Ben Moore, in gnuplot by grendel|khan. - Own work, Public Domain, https://commons.wikimedia.org/w/index.php?curid=31006154

Illumina Hiseq2000s BGI Hong Kong (128 units)

Moore’sCarlson’s

2001

peak Carlson

• Enabled by many

technologies

100k is 57 and 217

Page 28

Or put it another way, if computing had improved as fast….

IBM PC XT 1983 = 34 years ago

So for the technology we have now

the IBM PC would have been introduced in 2003

The same year of Nemo…

….or a car would cost €0.20… or a $1000 flight, 1 cent

Page 29

Market size and trends

• Lab instruments market

• $40bn (instruments/service)

• Segments and growth rate

• Oncology, infection, reproduction, agriculture, forensic, consumer

• Currently $3bn growing 30% CAGR to $12bn by 2022

• Market capitalisation:

• illumina (cap $28bn) make profit of $1.7bn on sales of $2.5bn

• Learnings for the vision supply chain:

• Rewards fall to the users, and system integrator

• Components suppliers get small % units sales

• Driver: cost per genome, not raw speed

• But someone has to learn the application and design the system

Grand view research; Macquarie (USA) Research 2014

Consumer genomics

Agri-genomics Forensics

Metagenomics, drug development

Immune system monitoring

Reproductive health

Clinical Investigation

Oncology

Page 30

Market trends and remaining opportunity

• Remaining potential for clinical applications

• On the cost reduction from $3.000M to $1000

• Moving WGS (Whole Genome Sequencing) into the doctors practice

• How many units? How many physicians? 10M

• Remaining potential for all other applications

• How much DNA is there out there?

Page 31

The next generation: Oxford nanopore

Page 32

What’s next? Following the trends

Page 33

Scientific and technological trends

• New science: Nobel prizes

• New imaging modes

• New labels

• New technology

• Sensors

• Optics

• Algorithms

• Robotics

• Drivers: faster, easy to use, more specific, sensitive, robust

Page 34




• New labels

• New technology

• Sensors

• Optics

• Algorithms

Evolution of the microscope

• Robotics


Page 35

Evolution of the microscope since c.1670

Expanding the market

• Drivers: quality, productivity by ease of use and automation

modern microscope imaging plate readervan Leeuwenhoek benchtop microscope

Individual cells: fluorescence brightfield

foldscope

plate of cells

Page 36

Automated cell counters: from $60k to $5k

Expanding the market

Beckmann, Invitrogen, SigmaAldrich

automated

manual

Page 37




• New labels

• New technology

• Sensors, Optics

• Algorithms

• Robotics


Page 38

Improved scientific knowledge: Nobel prizes for the lab

• The Nobel Prize in Chemistry 2008

• Osamu Shimomura, Martin Chalfie and Roger Y. Tsien†

• for the discovery & development of green fluorescent protein GFP

• New labels (antibodies, nanoparticles…)

• The Nobel Prize in Chemistry 2014

• Eric Betzig, Stefan W. Hell and William E. Moerner

• for the development of super-resolved fluorescence microscopy

• New imaging modes (Raman [1930], IR, spectroscopy…)

https://www.nobelprize.org/nobel_prizes/chemistry/laureates/2014/betzig.html

https://www.nobelprize.org/nobel_prizes/chemistry/laureates/2014/hell.html

https://www.nobelprize.org/nobel_prizes/chemistry/laureates/2014/moerner.html

Page 39

What is the resolution revolution ?

and why imaging is (still) important

Page 40



Ernst Abbe stated a limit

on resolving power (1873)

By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637

Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)

Page 41



Ernst Abbe stated a limit

on resolving power (1873)

By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637

Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)

Resolution scheme: adopted from Thorley et al., Super-resolution Microscopy: A Comparison of Commercially Available Options, Fluorescence Microscopy Super-Resolution and Other Novel Techniques, Academic Press, 2014

Page 42

Super-resolution: breaking the Abbe limit

Betzig et al., Science, 2006, 313, 1642-1645

to 20nm

to 50nm

Page 43

Super-resolution: how it works (1)

http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html

Page 44

New reagents: Brainbow labelling


Lichtman et al., Nature Reviews Neuroscience 2008

Page 45

Building brainbow from fluorescent proteins

• Motivation: to map all the connections in the brain

• What this means for the imaging supply chain:

• better faster smarter cameras

• multichannel, multifocal xyz-t-λ

Lawson Kurtz et al. / Duke University

Page 46




• New reagents

• New technology

• Sensors, optics

• Algorithms

• Robotics

• Lab environment


multiple

sensors

AndrewAlliance

Page 47

Smartlab at LabVolution Hannover, May 2017

Page 48

What the lab really looks like: it’s a messy place

A long way from

lean processes,

from industry 4.0

If the DNA is the

”job description

for the cell”, this

is what actually

happens when it

meets the worldCancer research makes for a messy bench. … @WorldwideCancer

Page 49

Applications tomorrow: watching the lab

• Klavins Lab

• “Aquarium”

• See TEDx talk on synthetic “programming” biology

https://www.youtube.com/watch?v=jL0cG4NJGd4

Page 50

Actions on future applications

• Future imaging (super-microscopes)

• Lab (factory) of the future: 20-100 cameras per lab

• Hospital of the future: role of imaging in the lab

• Take home message:

• imaging has proven value but still only present at a very low level

• Role of EU programmes

Page 53

Bringing it all together:

The Healthcare Lighthouse vision

Laboratory

Care

Surgery

RehabilitationeuRobotics topic groups on medical and laboratory robotics

Acknowledgements

• Almost too many to mention, but I’ll try

– DNA sequencing: Illumina, HPA, Qiagen

– Microscopy: PerkinElmer, Sartorius, Stefan Hell

– Cell counting: Luna, Roche, Jenoptik

– Smartlab: Deutsche Messe

– AndrewAlliance, EU, euRobotics

backup

Page 58

How much DNA is there out there?

6x1030

microbes

on earth

Page 60

Ebola and the most expensive tent in the world

Dr Sam Collins

Prof Ian Goodfellowactually an ion torrent machine

Page 61


http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html

Page 62


Localisation is

more precise

than resolving

In effect: trade

time for space

Role of imaging

Actually, several techniques http://www.practicallyscience.com/category/bio/cellbio/

"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection

Technology