Numerical Spectral Demulitplexing Microscopy of Measurements from an Anatomical Specimen Jason Deglint Vision and Image Processing Lab Farnoud Kazemzadeh Department of Systems Design Engineering Alexander Wong University of Waterloo David A. Clausi Waterloo, Ontario, Canada Abstract Multispectral microscopy is a method of capturing spectral bands using a microscope, and is used to observe specimens on a micron or nano scale. However, these systems are limited because they cannot capture transient phenomena since they cannot capture si- multaneous spectral information. We propose a new method called numerical spectral demultiplexing microscopy (NSDM) which uti- lizes a Raspberry Pi camera to capture RGB measurements and then infer narrow-band multispectral spectra. This is accomplished by training a non-linear regression random forest model based on the spectral sensitivity of the camera which allows for a low-cost, portable, and simultaneous capture multispectral microscopy sys- tem. We use the NSDM system as a bright-field multispectral mi- croscope and a dark-field fluorescence multispectral microscope on an anatomical specimen and show that additional information can be gathered by combining a bright-field and dark-field fluores- cence microscope. 1 Background Microscopes are optical instruments that allow the study and ob- servations of objects on the micron and nano scale. Most mi- croscopy systems capture spectral data in a limited number of wave- length bands which results in limited spectral information about the specimen. Moreover, when a multispectral microscope captures multiple spectral bands using a variety of light sources or optical filters, it is not done in a simultaneous manner, which prevents cap- turing transient phenomena when imaging samples in-vivo. These systems quickly become very large and expensive which limits their mobility and accessibility. To combat the issues that arise when sequentially capturing multispectral data, one approach is to capture RGB signals simul- taneously by using a consumer three-band RGB camera, and then inferring additional spectral information during the post-processing of the imaging data. We propose to use a new method to pre- dict narrow-band spectra from RGB measurements and use this method for microscopy applications. We call this method Numeri- cal Spectral Demulitplexing Microscopy (NSDM) which allows for a low-cost, portable multispectral microscope, which can be used for point-of-care application. 2 Methodology The NSDM system is an extension of previous work by Deglint et al. [1] and is broken into two components, as illustrated in Fig. 1. As the first component, we built a microscopy instrument using the Raspberry Pi camera system. To turn this camera into a digital microscope a cemented achromatic doublet lens with a focal length of 10 mm and a diameter of 8 mm was placed in the optical path of the Raspberry Pi camera. The second component is the numerical spectral demultiplexer, which is used to demultiplex the multiplexed RGB signals into a series of demultiplexed narrow-band imaging signals at different wavelengths using a non-linear regression model created from the prior knowledge of the spectral characterization of the detector. The spectral sensitivity of the camera was measured using a monochrometer from 420 nm - 720 nm at a resolution of 5 nm, resulting in 63 measurements. Using this spectral sensitivity, a forward model that mathematically describes the relationship be- tween 63 narrow-band wavelength signals and three multiplexed RGB imaging signals, can be written as M 3×1 = O 3×63 Λ 63×1 where M denotes the broadband multiplexed imaging measurements for the three different RGB channels and Λ represents the narrow- band spectral signals at 63 different wavelengths. The spectral characterization of the detector can be described by the observa- tion matrix, O. Since the inverse mapping Λ = O -1 M is non-linear, we em- ploy a non-linear regression model to model this mapping. The non-linear modeling approach used in this paper is a decision-tree strategy known as random forest modeling (RFM) [2]. (a) Bright-field Numerical Spectral Demulitplexing Microscope (NSDM) Fig. 1: A digital microscope was built using the Raspberry Pi cam- era and a simple lens. As seen, this microscope can operate as a bright-field microscope, or as a dark-field fluorescence microscope by placing a UV light source between the sample and the lens. The three channel broadband measurements of the bright-field or dark- field fluorescence microscope system are fed into the demultiplexer to predict narrow-band spectral signals. (a) Islet of Langerhans of the Pancreas Bright-field Image (b) Islet of Langerhans of the Pancreas Dark-field Fluorescence Image Fig. 2: The NSDM was used to capture images of the islet of Langerhans region of the pancreas using (a) a bright-field micro- scope and (b) a dark-field fluorescence microscope. In (a) different parts of the anatomical specimen have similar predicted spectral signatures. In (b) the same pixel locations have much different pre- dicted spectral signatures and therefore can be distinguished. 3 Results and Discussion The transmission spectra at two different pixels of the islet of Langer- hans region can be seen in Fig. 2a. Comparing these two spec- tral signatures demonstrates that they are very similar in nature. However, if we inspect the same pixels in the reflection spectra of the dark-field fluorescence image (Fig. 2b), we can see that the pixels predict significantly different signatures. This illustrates that additional information can be gathered by combining a bright-field NSDM system with a dark-field fluorescence NSDM system. References [1] Deglint, J., Kazemzadeh, F., Shafiee, M. J., Li, E., Khodadad, I., Saini, S. S., Wong, A., and Clausi, D. A., “Virtual spectral multiplexing for applications in in-situ imaging microscopy of transient phenomena,” in [SPIE Optics and Photonics ], (August 2015). [2] Breiman, L., “Random Forests,” Machine Learning (2001).
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Numerical Spectral Demulitplexing Microscopy of Measurements from an Anatomical SpecimenJason Deglint Vision and Image Processing LabFarnoud Kazemzadeh Department of Systems Design EngineeringAlexander Wong University of WaterlooDavid A. Clausi Waterloo, Ontario, Canada
AbstractMultispectral microscopy is a method of capturing spectral bandsusing a microscope, and is used to observe specimens on a micronor nano scale. However, these systems are limited because theycannot capture transient phenomena since they cannot capture si-multaneous spectral information. We propose a new method callednumerical spectral demultiplexing microscopy (NSDM) which uti-lizes a Raspberry Pi camera to capture RGB measurements andthen infer narrow-band multispectral spectra. This is accomplishedby training a non-linear regression random forest model based onthe spectral sensitivity of the camera which allows for a low-cost,portable, and simultaneous capture multispectral microscopy sys-tem. We use the NSDM system as a bright-field multispectral mi-croscope and a dark-field fluorescence multispectral microscopeon an anatomical specimen and show that additional informationcan be gathered by combining a bright-field and dark-field fluores-cence microscope.
1 BackgroundMicroscopes are optical instruments that allow the study and ob-servations of objects on the micron and nano scale. Most mi-croscopy systems capture spectral data in a limited number of wave-length bands which results in limited spectral information about thespecimen. Moreover, when a multispectral microscope capturesmultiple spectral bands using a variety of light sources or opticalfilters, it is not done in a simultaneous manner, which prevents cap-turing transient phenomena when imaging samples in-vivo. Thesesystems quickly become very large and expensive which limits theirmobility and accessibility.
To combat the issues that arise when sequentially capturingmultispectral data, one approach is to capture RGB signals simul-taneously by using a consumer three-band RGB camera, and theninferring additional spectral information during the post-processingof the imaging data. We propose to use a new method to pre-dict narrow-band spectra from RGB measurements and use thismethod for microscopy applications. We call this method Numeri-cal Spectral Demulitplexing Microscopy (NSDM) which allows for alow-cost, portable multispectral microscope, which can be used forpoint-of-care application.
2 MethodologyThe NSDM system is an extension of previous work by Deglint etal. [1] and is broken into two components, as illustrated in Fig. 1.As the first component, we built a microscopy instrument using theRaspberry Pi camera system. To turn this camera into a digitalmicroscope a cemented achromatic doublet lens with a focal lengthof 10 mm and a diameter of 8 mm was placed in the optical path ofthe Raspberry Pi camera.
The second component is the numerical spectral demultiplexer,which is used to demultiplex the multiplexed RGB signals into aseries of demultiplexed narrow-band imaging signals at differentwavelengths using a non-linear regression model created from theprior knowledge of the spectral characterization of the detector.
The spectral sensitivity of the camera was measured using amonochrometer from 420 nm - 720 nm at a resolution of 5 nm,resulting in 63 measurements. Using this spectral sensitivity, aforward model that mathematically describes the relationship be-tween 63 narrow-band wavelength signals and three multiplexedRGB imaging signals, can be written as M3×1 = O3×63Λ63×1 whereM denotes the broadband multiplexed imaging measurements forthe three different RGB channels and Λ represents the narrow-band spectral signals at 63 different wavelengths. The spectralcharacterization of the detector can be described by the observa-tion matrix, O.
Since the inverse mapping Λ = O−1M is non-linear, we em-ploy a non-linear regression model to model this mapping. Thenon-linear modeling approach used in this paper is a decision-treestrategy known as random forest modeling (RFM) [2].
(a) Bright-field Numerical Spectral Demulitplexing Microscope (NSDM)
Fig. 1: A digital microscope was built using the Raspberry Pi cam-era and a simple lens. As seen, this microscope can operate as abright-field microscope, or as a dark-field fluorescence microscopeby placing a UV light source between the sample and the lens. Thethree channel broadband measurements of the bright-field or dark-field fluorescence microscope system are fed into the demultiplexerto predict narrow-band spectral signals.
(a) Islet of Langerhans of the Pancreas Bright-field Image
(b) Islet of Langerhans of the Pancreas Dark-field Fluorescence Image
Fig. 2: The NSDM was used to capture images of the islet ofLangerhans region of the pancreas using (a) a bright-field micro-scope and (b) a dark-field fluorescence microscope. In (a) differentparts of the anatomical specimen have similar predicted spectralsignatures. In (b) the same pixel locations have much different pre-dicted spectral signatures and therefore can be distinguished.
3 Results and DiscussionThe transmission spectra at two different pixels of the islet of Langer-hans region can be seen in Fig. 2a. Comparing these two spec-tral signatures demonstrates that they are very similar in nature.However, if we inspect the same pixels in the reflection spectra ofthe dark-field fluorescence image (Fig. 2b), we can see that thepixels predict significantly different signatures. This illustrates thatadditional information can be gathered by combining a bright-fieldNSDM system with a dark-field fluorescence NSDM system.
References[1] Deglint, J., Kazemzadeh, F., Shafiee, M. J., Li, E., Khodadad,
I., Saini, S. S., Wong, A., and Clausi, D. A., “Virtual spectralmultiplexing for applications in in-situ imaging microscopy oftransient phenomena,” in [SPIE Optics and Photonics ], (August2015).
[2] Breiman, L., “Random Forests,” Machine Learning (2001).
Related Documents
