11. Real-Time Processing System Based on FPGA for Electronic Endoscope

8/3/2019 11. Real-Time Processing System Based on FPGA for Electronic Endoscope

1/4

Real-time Im age Processing System Based on FP GAfor Electronic E ndoscope

Jie Jiang Daoyin Yu Zheng SunSchool of Precision Instrument and @to-electronic Engineering, Tianjin University,

Tianjin, 300072, ChinaK Lab of Opto electron ic Inform ation Science and Technology, MOE ,Tianjin, China

E-mail: [email protected] Abstract: Medical Electronic Endoscope is theoutcome of combining traditional endoscopetechnology with modern computer technologyand micro-electronic technolow. Because of thelimit of its space endoscope adopts a kind ofcompact full pa me CCD whose output imagepattem is diflerent f i m the pattern of PALmonitor And to improve image quality anddisplaying eflect the system implements real-timeimage processing with field programmable gatearray (FPGA). The image patterntransformation ensures the correct display of theCCD output ima ge. With the function of imageinterpolation zooming, image enhancement anddigital video technolow, image quality and theaccuracy rate of diagnosis are improved.

I IntroductionElectronic Endoscope- kind of medical

instrument is used widely. Through endoscopedoctors can duectly visualize internal a reas ofthe human body, diagnose quickly and do someminiature injury surgery. Doctors and patientsrecognize the advantages of ElectronicEndoscope. Electronic Endoscope is theoutcome of combining traditional endoscopetechnology with modern computer technologyand microelectronic technology. From 198O's,with the development of Electron ic Endoscope,the image processing fo r E lectronic Endoscopedevelops fast. J.F.Rey captured the video signalfrom Electronic Endoscope into computer forimage processing and analyzing. S.Guadagniimplemented an im age processing and analyzingsystem with an Electronic Endoscope and a 386computer. Japan develops and producesElectronic Endoscope as well as image

processing system for Electronic Endoscope ingreat number.

Because of the limit of its space ElectronicEndoscope adopts a kind of compact full frameCCD whose output image pattem is differentfrom the pa ttem of PAL monitor. And to imp roveimage quality and displaying effect the systemimplements real-time image processing withfield programmable gate array (FPGA). FPGA isadopted for its short development cycle, in-system programming, cost-effectiveness, high-density logic integration and high performance.I1 Performance and Configuration of

the Real-timeImage Pro cessing System

The function of CCD im age acquisition isimplemented. But the signal from CCD is imagepackets shown as th e f i g 1 which is differentfrom the image pattem ofPAL monitor shown asthe fig.2.. . i

- . . . \Hori-Antal syn f Image signalFig. 1 Imag e signal from CC D

CCD output signal can not be displayedcorrectly on PAL monitor because of th edifferent between their horizontal frequency,fram e frequency and synchronization signal.+9" CCD is built in the distal end of endo-

Limited by digestive tract, endoscope is about

0-7803-6253-5/00/$10.002000 EEE. 682

8/3/2019 11. Real-Time Processing System Based on FPGA for Electronic Endoscope

2/4

+. . ,* I . ......... ,, > ........I . . - ......... ...... >. .,. ..... , . .

. , * . . a . . . .. ....!...: \; ......... * ..Image si Light integral Fig. 2 Video signal of PAL

scope, so the size of CCD s also limited, only190X 190 pixels. The picture is so small thatdoctors can watch clearly. So it need realize areal-time digital interpolator.

Because of reflection of bead in digestive tractand different luminance, image is blurry. So theimage enh ancement is necessary.

To improve the displaying effect, some digitalvideo technology is implemented: mult-pictureeffect and image freezing. The shape andlocation of pictures are control too for beauty.

As described above, the real-time imageprocessing system performs the function ofimage pattern transformation, imageinterpolation zooming, image enhancem ent anddigital video techno logy etc.

The figure above illustrates that the real-timeimage processing system is implemented withfirst-in-first-out (FIFO) buffer and video-double-port memory (VRAM) cooperating with FPGA.The implementation of the real-time imageprocessing system is described in detail asfollows.

,+- -.

I........

Fig. 3 The diagram of real-time imageprocessing system

m T he Design and Implementationwith FPGA of Real-time

Image Processing System3.1 Image Pattern Transformat ion

There is still a long d istance between the CCDoutput image pattern and the requirement ofpatterns w hich PAL mo nitor can display mainlybecause of the difference between theirhorizontal frequency, frame frequency andsynchronization signal.

CCD s horizontal frequency, app roximately30 kHz, is twice as much as the line scanfrequency (15.625 WZ) required by PAL. Toresolve the discrepancy between two horizontalfrequencies the system setups a FIFO buffer tobuffer the CCD output signal.

Because of using full frame CCD whichoutputs signals line by line and outputs oneframe signal in one field time, frame frequency,equal with field frequency, is 50 Hz and PALSframe scan frequency is 25 Hz adoptinginterleaved scan. The system removes theinconsistencyof frame frequency by partitioningVRAM. After such man-m ade separation of oddand even field, a line-by-line CCD mage can be&splayed on the monitor correctly. Both read &write control and address generation of FIFOand VARM are implemented with FPGA.

CCD output image signals havent anysynchronization while PAL image correctdisplaying needs synchronization as its timenorm. The system generates synchronizationbased on principle of phase-lock loop (PLL) an dadds synchronization signal to CCD outputimage by video encoder to ensure the imagedisplaying on PAL monitor correctly.3.2Image Interpolation Zooming

The Walsh method is adopted in imagezooming from many interpolation algorithms forit is simple, intuitionistic. It can easily beimplemented with hardware too. Interpolationzooming algorithmic is based on superimposurefilter which is put forward by Wang Zhaohua.Because it overcomes edge effect, the imagequality is good. In order to simplify the hardware,the zooming template is decomposed and

683

8/3/2019 11. Real-Time Processing System Based on FPGA for Electronic Endoscope

3/4

implemented with simple arithmetic of FPGAlogic circuit.

Observing the image. [M,] = - 2 4 2 for interpolation, for4[ 2 1I example of 4 fold

zooming, we find out that in convolution zerosdo not take part in compute, so we decomposethe template of 4 fold zooming(3 X 3 matrix)which is showed as above into four simpletemplates which is shown as follow:

which accomplishes enhancing and zoomingsimultaneously .The new template shown asfig.5, after being decomposed, can be easily

0 - 1 1 4 -112 -114 0- 114 114 1 1 / 4 - 1 1 4-112 1 3 1 - 112-114 11 4 1 114 -1140 - 114 -112 -114 0

Fig. 5 Walsh enhancementzoomina temdate1

implemented with hardware. Walshenhancement zooming template reducescomputing procedures and time and simplifies~.From above, we know that the original pixel hardw are And it can also be

is remained, the other three pixels areinterpolated with four neighboring pixels. Thespeed of the algorittim is 4 times faster thanconvolution. And the algorithm can beimplemented with simple ad dition or subtractionoperation.All of these make the implementationin bardware is not only simple but also fast.Through ob servation, all zooming templates canbe decomposed, implemented with simpleoperation in this way. The interpolated image hasgood quality and hasnt jagged lines. Fig. 4 isthe block diag ram of 4 fold zoom ing processing

decomposed and implemented with simplearithmetic, For such reasons it satisfies ourreal-time hardware implementation very much.On the basis of interpolation zooming real-time image enhancement zooming isimpfementedwith FPGA.3.4 Digital Video Technology

- Multi-picture effectThe system displays two pictures: the small

one is CCD output image and the large one isobtained by zooming 4 times CCD output image.Fig 6 shows the effect of double-p icture.

with FPGA.

II

,1. . - __ . _. Fig.6 Double-picture effect

Fig. 4 Block diagra m of 4 fold zoomingprocessing with FPGA.

3.3 Image EnhancementFor the purpose of simp llfying system as well

as being easy to the hardware implementationthe system s till adop ts Walsh method to enhanceimages. During experiment after combining thetemplate of zooming and enhancement, a newenhancement zooming template is obtained

From the figure we can find that the twopictures shapes are controlled by cutting eachones four edges for the purpose of beingaesthetic and every pictures position is car ehl lychosen to reach the best d isplayin g effect. All ofthis is implemented with FPGA imageinterpolation zooming and VRAM read & writecontrolledby P G A .* lmage fmezing

The image that contains probable focus is

684

8/3/2019 11. Real-Time Processing System Based on FPGA for Electronic Endoscope

4/4

frozen for do ctors to diagnose further. Of the twoimages one is real-time displaying and the o theris frozen. Showing states of them can betransformed: real-time one can be changed to befrozen and vice versa. Two switches aredesigned to perform real-time control of thetransformation of pictures.N Experiment R esults

The experiment results of interpolationzooming and enhancement zooming is shownbelow:

(a) Original image

(b) 4 Times zooming image

(c ) 4 Times enhancement zoom ing imageFig.7 Experiment Results

The figures illustrate that after interpolationzooming the image without the edge indentionand block effect is obtained. The quality of

image is better. With enhancement zooming, thedefinition of images is improved greatly whichcan be helpful to doctors diagnose.

V ConclusionThe performance and configuration of real-

time image processing system for ElectronicEndoscope are presented above. The design andimplementation with FPGA of image pattemtransformation, image interpolation zooming,image enhancement and digital video technologyare described in detail, too. The real-time imageprocessing has been used in ElectronicEndoscope successfully. The image quality andthe accuracy rate of diagnosis are improved.

FPGA is adopted as the core of the system fo rits changeable and reconfigumble feature, so thesystem can be easily modified, moreover theperformance of the system can be improved inthe future, without the change of the hardware.With a large amount of gates in a single device,FPGA provides the system with high integrationand excellent performance.

ReferencesS. M. Greengrass & M. Cunningham,Endoscopy, Measurement+Control, Vol. 26,1993: 109-114M. Classen, etc. al., Electronic Endoscopy-The Latest Technology, Endoscopy, Vol.19,198 7: 118-123S.Guadagm, etc. al., Imaging in Dg estiveVideo Endoscopy, SPIE Vol. 14 20, Op ticFibers in Medicine, 1991: 178- 182Z.Wang, Theoretical analysis andexperimental investigation of the two-lmensionai spectrum of a PAL compositevideo signal, ntzArchir Bd.5 1983 H.5Nixos Herodotou and AnastasiosN.Venetsanopoulos, COLOUR IMAGEINTERPOLATION FOR HIGHRESOLUTION AXQUISITION ANDDISPLAY DEVICEJEEE Tran. onConsumer Electronics, Vol.Jl,N0.4 ,1995,1111118-1125

685