Mobile Microscopy ECE 445 Design Review Group #33 Names: Rui Guo, Xiaoyu Qin, Yongli Chen TA: Ben Cahill February 2015 1
Mobile Microscopy
ECE 445
Design Review
Group #33
Names: Rui Guo, Xiaoyu Qin, Yongli Chen
TA: Ben Cahill
February 2015
1
Tables of Contents
1.0 Introduction 1.1 Statement of Purpose…………………………………………………………………3 1.2 Objective
1.2.1 Goals & Benefits…………………………………………………………….3 1.2.2 Functions & Designs………………………………………………………...3
2.0 Design 2.1 Block Diagram………………………………………………………………………...4 2.2 Block Description
2.2.1 Data collection/preprocessing……………………………………………….7 2.2.2 Data Processing and Software Designing…………………………………...7 2.2.3 DPM system………………………………………………………………....7 2.2.4 iOS MVC……………………....…………………………………………....7 2.2.5 Microscope Port……………………......……………………….…………...8 2.2.6 Media Processing..…………………......……………………….…………...8
2.3 Schematics 2.3.1 Microcontroller with LED setup…………………………...……..…….…...9 2.3.2 Optical components……….……………………………………….……....12
2.3.2.1 Microscope & DPM Port Setup………………………………….13 2.3.2.2 Alternative Microscope Configuration…………………………..14
3.0 Requirements and Verification Table of Requirements, Verification, and Tolerance…………………………………….14
4.0 Tolerance Analysis Tolerance Analysis……………………………………………………………………….16
5.0 Cost & Schedule 5.1 Cost Analysis
5.1.1 Labor……………………………………………………………………….16 5.1.2 Parts………………………………………………………………………...17 5.1.3 Grand Total………………………………………………………………...17
5.2 Schedule…………………………………………………………………………...…18 6.0 Ethics
Ethics...………………………………………………………………………….………..20 7.0 Safety
Safety Statement...…...………………………………………………………….………..21 8.0 Reference
Reference………………………………………………………………………………...22
2
1.0 Introduction
1.1 Statement of Purpose
In medical areas, doctors always need to examine patients’ cell growth in order to keep
track of their conditions. However, retrieving the quantitative phase of cell image with high
sensitivity is imposed by phase noise due to mechanical vibration and air fluctuation. DPM
(Diffraction Phase Microscopy) is a common path imaging method that alleviates the noise
problem. It will be convenient for doctors to analyze the cells simply by mobile phone.
Furthermore, mobiles nowadays have powerful cameras and people won’t need to buy a separate
digital camera. Most importantly, smartphone offer much smaller sensors in terms of resolution,
and have smaller lenses. Therefore, importing the DPM on mobile platform and test how
microscopy system works on mobile phone cameras. It is anticipated to see how it will benefit
the scientific needs.
1.2 Objectives
1.2.1 Goals & Benefits:
Easy and stable setup that users can bring anywhere
Processing of photos using iPhone completely
Various magnification toggles from 10X to 100X
Record DPM processed cells and generate relevant result
1.2.2 Functions & Designs:
Self provided strong light used for microscope light source
All components mount on iPhone, and easily removable
Only button battery needed, saved time for user to plug in Vcc.
Simple switch to turn microscope on and off
Lightweight and compact design for users to carry
Friendly GUI for user to analyze and capture cells motions
3
2.0 Design
2.1 Block Diagram
Figure 1. TopLevel System Layout
As shown in Figure 1, the left box is mainly our electrical and optical components for this
project, the right box is the programming part for our project.
For the left box, the cell sample is illuminated by the LED then going through the microscope
port for magnification. The output of the microscope will feed into the DPM system, to reduce
noise for better resolution. Then it will feed into the iPhone’s camera port.
Figure 2. DPM System Schematics
As shown in Figure 2. This is the DPM block diagram for our project. it reduces noise by
filtering signal through the 4f pinhole filter. The result will have a better resolution.
We are building the DPM system. It should not have a big magnification. 2.5 will be the best.
Otherwise, the image will not be resolved by the grating.
We can find the best magnification by the lens make equation(e.q.(1)):
In total, the optical system consists of 100x magnification (40x for the microscope part, 2.5x for
the DPM system).
4
This is the detailed flowchart for the signal reconstruction that has to be implemented.
Figure 3. Phase Reconstruction Procedure
In this step of programming, we are implement the following equations (2D FT & 2D Inv FT):
Fast Fourier Transform will save us a lot of time in computing, that’s why we are using OpenCV
framework because it is written in C++ and it has the best performance in image processing.
5
A flowchart of segmentation(unwrapping):
Figure 4. Flow chart of the unwrapping step
Figure 4 explains most of the work in segmentation. Basically, we want to do some segmentation
in the spectrum we generated. 0th order should always be excluded because it is DC part and it
doesn’t contain any useful signal. What we really need is the first order part of the spectrum, so
we need to shift and center the spectrum.
6
2.2 Block Description
2.2.1 Data collection/preprocessing
The data processing part should provide the image from the cell sample. Through the
microscope, it can generate a magnification of the cell sample from 10X to 100X. Then DPM
system is applied to improve the image result. Afterward, the result graph will be fed into iPhone
camera for application usage.
2.2.2 Data Processing and Software Designing
In this part, the application are reading the photos taken from camera. Then it will
process each photo, reducing the background noise by DPM algorithm. Afterwards, the
application will display the processed result to the phone screen for user’s need.
2.2.3 DPM system
In general, it’s a 4f system that diffracts the microscope output image with a grating as
well as obtaining a background image of the sample. Subtraction the cell image and background
image at fourier domain will eliminate noises as much as possible.
A diffraction grating is needed in conjunction with 4f system in order to produce
interference and obtain the phase information from the scattered fields. It is placed at the output
image plane of the microscope. which will create copies of the image at different angle.
Under 4f system, the first lens takes a Fourier Transform, creating a Fourier plane. After
1st order filed filtering down using a small pinhole, full 0th field and partial 1st order field takes
Fourier Transform again and becomes a uniform plane wave. They interfere with each other to
create the interferogram at the CCD plane. The pinhole is the spatial filter used at the Fourier
plane. It allows full 0th order spectrum to pass but filters down 1st order spectrum.
2.2.4 iOS MVC
MVC(model view controller) is a software architectural pattern for implementing user
interface. Xcode will be the major tool for programming. It is a software made for Apple
developers and can be downloaded as a free software on Mac OS X. The version now is 6.1.1.
7
2.2.5 Microscope Port
Both the scattered and unscattered fields are captured by the objective lens and focused
on its back focal plane. A beam splitter then redirects the light through a tube lens, creating a
collimated beam containing the image at the output image plane of the microscope. This is where
a camera is typically placed in order to get the intensity images, but in DPM we require phase
images, so interference should be performed at this place.
2.2.6 Media Processing
The iPhone will internally store an image with only the background. It will read the photo
taken and process it according to the DPM procedure and generate a reconstructed unwrapped
phase (Obtained by Hilbert Transform).
OpenCV will be the main source library of this project. OpenCV was designed for
computational efficiency and with a strong focus on realtime applications. It has its framework
on iOS devices. Some of the frameworks that Apple provided will be used too. The major
frameworks are AVFoundation, AppKit and Accelerate Framework.
8
2.3 Schematics
2.3.1 Microcontroller with LED setup
Figure5. Top view for LED light source array
This light source is designed to provide proper visibility for microscope. There are a total of 9
LEDs, and each LED is white LED. All LEDs on the same concentric circle will share a single
switch. There are four levels of brightness:
1. All LEDs are off. (0 LED is on)
2. The central LED is turned on. (1 LED is on)
3. The central LED is on, and the middle LEDs are on. (4 LEDs are on)
4. All LEDs are on. (9 LEDs are on)
9
The above diagram is schematic for LED array circuit used for light source of microscope. There
are three major components: switches, Arduino microcontroller and LED array.
Firstly, switches are designed for users to control light intensity. With two DIP switches, we
have 22 = 4 levels of intensity. If we use 0 for off and 1 for on, then 00 stands for no LED is on,
01 stands for 1 LED is on, 10 stands for 4 LEDs are on, and finally 11 stands for all 9 LEDs are
on. Moreover, with more switches we will be able to control more LEDs. All LEDs are white
LEDs since they are used for light source.
Secondly, Arduino UNO microcontroller is used for decoding the 2bit signal from switches and
providing current control for base terminals of bipolar junction transistors. When output pin from
Arduino gives high (+5v) voltage, it provides approximately ~0.5 mA. Combining with a DC
current gain of 2N2222 NPN amplifier, LED is turned on.
Thirdly, LED array component contains 3 resistors for base terminals, 9 LEDs with one resistor
for each, and 3 bipolar junction transistors. The 3 1kΩ resistors for base terminals are used for
limiting current through base terminals. The resistors following LEDs are used for limiting
current through LEDs. All the other parts are explained above. The white LED has a typical turn
on voltage of ~3.3V, and current of ~20mA. The 100Ω resistors following LEDs can limit the
currents through LEDs around ~17mA, which is enough for powering the LEDs.
11
2.3.2 Optical components
2.3.2.1 Microscope & DPM Port Setup
Fig 7. Microscope, DPM & iPhone Setup
12
Figure 7 is the design for the optic components, it includes the microscope(dotted box)
and the DPM system on the right.
The magnification for the microscope is 40x. Constructed by using two lenses with focal
length of 5mm and 200mm. Using the formula:
M | f2/f1 00/5 40 | = = 2 =
The output image will then undergo the DPM for filtering. The DPM system consists of
the diffraction grating, the Fourier plane and two lenses with focal length 60mm and 150mm.
Two lenses will combine to make a magnification of 2.5x. Final image will be feed into the
iPhone camera.
2.3.2.2 Alternative Microscope Configuration
Fig 8. Microscope System
In addition to the homebrew version of the designed version of the microscope. A
namebrand microscope can be used to observe the cell sample as well. What we used in the
project is the Zeiss Z1 electron microscope with a magnification of 40. This alternative setup can
also help in developing the homebrew microscope. To verify the correctness of the design.
13
3.0 Requirement and Verification
Module/Part Name
Requirement Verification
Lens
Optimal range of different types of lens: Tube lens: 200 40 mm± Objective lens: 10 2 mm± M1 = 40 DPM lens: 200 40 mm (second± larger than the first) M2 = 2.5
To test if the focal length fits project’s need, we need to measure it. The procedure is: Put the lens parallel with a wall or white flat surface and focus distant light source (streetlam), horizon or lamp in your room on the wall until you get sharp image. The lamp must be near optical axis of the lens. a is distance to lamp, streetlamp, or horizon. a' is distance to wall.
Diffraction Grating
Grating period should be suitable for iPhone resolution. Typically, 9
.mμ Diffraction angle should be >30 degree
Test the gratings in the lab bench first. Check how green laser light is diffracted at the CCD plane and calculate the diffraction angle given the measured data.
Tube To make a stable DPM, we need external tube to support the system. Tube has to be designed so that there is only internal reflection. Best material to find is optical fiber. Length: 5 0.5 m±
Test tube made with different materials. Compare the image of cell sample by comparing brightness and contrast.
Power 5V +/ 0.25V for button cell.
Measure the battery time for button cell
Lens II Numerical Aperture It is a dimensionless number that characterizes the range of angles. Defined as . where n is the refractive index. The NA requirement for lens should be as least: 0.53 ± 0.05
From a point source, measure the half angle lens can accept. Then look up the refractive index that given in the datasheet. Calculate for the NA.
14
Pinhole (Spatial Light Modulator)
Diameter: 10 1.5 µm.± Two holes: small 1st order, large 0th order
Use ruler to measure the exact diameter. Test at lab bench how much light accepted by 0th order (as much as possible), how much light covered by 1st order.
Clip/Structural Holder
For structural design, DPM system needs to connect directly to iPhone. Therefore, a connector (i.e. a clip) is needed.
Test the intensity of the holder by how much weight it can hold.
Resistor The resistor’s tolerance is: 100Ω ± 5%, 1/8Watt 1000Ω ± 5%, 1/2Watt
Use a steady 3V provided by the power supply and apply it on the resistor, then use a multimeter to read the current through the resistor, The resistance will be calculated through Ohm’s Law.
Arduino UNO R3 board with DIP ATmega328P
Power supply voltage: 712V Power supply current lower limit: 50mA Output current limit: 450mA
Use an input voltage of 9V to power the Arduino UNO. Then connect pin 11, 12 to switches, and connect pin 5,6,7 to 3 base resistors, which are followed by base terminals of bipolar junction transistors(2N2222) . Then test pin 5,6,7’s output based on various combination of pin 11&12’s input. If working as desired, then the program written is correct.
15
4.0 Tolerance Analysis
Lens:
Diameter tolerance: it is a very important mechanical tolerance that must be considered if
the optic is going to be mounted in any type of holder. Typical manufacturing tolerances
for diameter are: +0.00/0.10 mm for typical quality, +0.00/0.050 mm for precision
quality, and +0.000/0.010 mm for high quality.
Center Thickness Tolerance: Center thickness is measured across the mechanical axis of
the lens, defined as the axis exactly between its outer edges. Variation of the center
thickness of a lens can affect the optical performance because center thickness, along
with radius of curvature, determines the optical path length of rays passing through the
lens. Typical manufacturing tolerances for center thickness are: +/0.20 mm for typical
quality, +/0.050 mm for precision quality, and +/0.010 mm for high quality.
5.0 Cost and Schedule
5.1 Cost analysis
5.1.1 Labor (in U.S. dollar $)
Name Rate ($/hr) Total working hours Total
Yongli Chen 30 250 $15000
Rui Guo 30 250 $15000
Xiaoyu Qin 30 250 $15000
Total 90 750 $45000
16
5.1.2 Parts
Name Manufacturer Quantity Cost
Lens(40150mm, f/4.05.6 )
Olympus 2 $238
Lens(5mm, H0514MP2)
Computar 1 $159
Lens(200mm) Canon 1 $699
White LED #L11W5TH151
ledsupply.com 10 $5
iPhone 5s Apple 1 $539.99
Pinhole(SLM) 2
Diffraction grating Rainbow Symphony
2 $14.99
iPhone 5s camera mount Amazon 1 $23.89
Wall adaptor power supply (9V, 650mA)
Amazon 1 $5.65
Arduino UNO R3 board with DIP ATmega328P
Amazon 1 $25.85
Resistor 1000Ω Radioshack 5 $1.49
Resistor 100Ω Radioshack 10 $3.98
Total $1176.87
5.1.3 Grand Total
Name Total
Labor $45000
Parts $1176.87
Grand Total $46176.87
17
5.2 Schedule
Week Tasks Responsibility
Jan 20 Discuss possible topics,initial posts All
Jan 26 Finalize topic for the project All
Feb 2 Finish writing up project for approval Research for basic documents writing up
Rui Guo, Xiaoyu Qin Yongli Chen
Feb 9 Finalize proposal Intro & Design Requirement & Verification Cost analysis, formatting, index
Yongli Chen Rui Guo Xiaoyu Qin
Feb 16 Begin initial software development Mock design review
Rui Guo Xiaoyu Qin, Yongli Chen
Feb 23 Finish initial software(interfaces, data formatting)
Official Design Review Laboratory safety training
Rui Guo Yongli Chen, Xiaoyu Qin all
March 2 Begin initial hardware development (microscope)
Sorting necessary parts Soldering assignment
Yongli Chen, Rui Guo Xiaoyu Qin All
March 9 Hardware development continue (microscope)
Testing and fitting
Rui Guo, Yongli Chen Xiaoyu Qin
March 16 Hardware fitting on iPhone’s rear camera Circuit model for microscope Circuit power model
Xiaoyu Qin Rui Guo Yongli Chen
March 23 (Spring Break)
Finish Hardware development (microscope)
Analyze data type for software Software development
Rui Guo Xiaoyu Qin Yongli Chen
March 30 Continue on software development (Analyzing data)
Data processing Mobile software testing
Yongli Chen Rui Guo Xiaoyu Qin
18
April 6 Software development on using Apple’s api to retrieve photos
Image processing Test the software on iPhone
Yongli Chen Rui Guo Xiaoyu Qin
April 13 Software development on image processing
Cell Recognition Analyzing cell’s state
Rui Guo Xiaoyu Qin Yongli Chen
April 20 Debugging hardware finishing software development Prepare Final Paper & presentation
Xiaoyu Qin Yongli Chen Rui Guo
April 27 Assemble the microscope and run the program to further debugging
Finalize demo Prepare Final Paper & presentation
Yongli Chen Xiaoyu Qin Rui Guo
May 4 Finish Final Paper and presentation Final Paper Presentation
Rui Guo, Xiaoyu Qin Yongli Chen, Xiaoyu Qin
19
6.0 Ethics
In making of this mobile microscope, we all commit with the IEEE code of ethics. The related ones are listed below:
1. to accept responsibility in making decisions consistent with the safety, health, and welfare of the public, and to disclose promptly factors that might endanger the public or the environment;
2. We will make sure there will be no potential damage to the safety and health of the person using the device
3. to be honest and realistic in stating claims or estimates based on available data; 4. We will be honest with our data and claims, based on our own experiments 5. to improve the understanding of technology; its appropriate application, and potential consequences; 6. to maintain and improve our technical competence and to undertake technological tasks for others only if qualified by training or experience, or after full disclosure of pertinent limitations; 7. to seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors, and to credit properly the contributions of others;
20
7.0 Safety Statement
Emergency Call 911, address of ECEB is 306 N Wright St, Urbana, IL 61801
Laser
For our design the laser we will be using are visible lasers. From 400780 nm in wavelength. Depending on the intensity of the laser, it can possibly cause “photochemical damage to the retina or retinal burn.” Goggles should be wearing throughout the development process. Do not damage other people in the lab. Be extra caution when using laser
Electronic safety Always make sure you are not alone inside the lab
Health
For your own safety and others. Do not go to the lab if you are sick, and sanitize your hand after going to the lab.
Fire alarm When fire alarm rings, turn off all the electronics you are using in the lab and leave the
building through emergency exit.
21
8.0 Reference
1. B. Bhaduri, C. Edwards, R. Zhou, H. Pham, T. H. Nguyen, L. L. Goddard and G. Popescu
Diffraction phase microscopy: principles and applications in materials and life sciences,
Advances in Optics and Photonics, 6, 57 (2014)
2. B. Bhaduri, H. Pham, M.Mir and G. Popescu, Diffraction phase microscopy with white light,
Opt. Lett., 37 (6), 1094 (2012).
3. Greivenkamp, John E. (2004). “Field Guide to Geometrical Optics”. SPIE Field Guides vol.
FG01. SPIE. ISBN 0819452947. p. 29.
4. Schottner, G (May 2003). "Scratch and Abrasion Resistant Coatings on Plastic Lenses—State
of the Art, Current Developments and Perspectives". Journal of SolGel Science and
Technology. pp. 71–79. Retrieved 28 December 2009.
5. Radioshack, “1K OHM 1/2W 5% CARBON FILM RESISTOR PK/5”, datasheet. [Online].
Available:http://www.radioshack.com/1kohm12w5carbonfilmresistorpk5/2711118.html
[Accessed March 1, 2015].
6. Radioshack, “1/8WATT 100 OHM CARBON FILM RESISTORS”, datasheet. [Online].
Available:http://www.radioshack.com/18watt100ohmcarbonfilmresistors5pack/271000
5.html [Accessed March 1, 2015].
7. LEDsupply, “5mm LED White 15 Degree Viewing Angle”, datasheet. [Online].
Available:http://www.ledsupply.com/leds/5mmledwhite15degreeviewingangle [Accessed
March 1, 2015].
8. ARDUINO, “Arduino Uno R3”, datasheet. [Online].
Available:http://arduino.cc/en/Main/arduinoBoardUno [Accessed March 1, 2015].
22