INTRODUCTION Image fusion is the process by which two or more images are combined into a single image retaining the important features from each of the original images. The fusion of images is often required for images acquired from different instrument modalities or capture techniques of the same scene or objects. Important applications of the fusion of images include medical imaging, microscopic imaging, remote sensing, computer vision, and robotics. Fusion techniques include the simplest method of pixel averaging to more complicated methods such as principal component analysis and wavelet transform fusion. Several approaches to image fusion can be distinguished, depending on whether the images are fused in the spatial domain or they are transformed into another domain, and their transforms fused. With the development of new imaging sensors arises the need of a meaningful combination of all employed imaging sources. The actual fusion process can take place at different levels of information representation, a generic categorization is to consider the different levels as, sorted in ascending order of abstraction: signal, pixel, feature and symbolic level. This focuses on the so-called pixel level fusion process, where a composite image has to be built of several input images. To date, the result of pixel level image fusion is considered primarily to be presented to the human observer, especially in image sequence fusion (where the input data consists of image sequences). A possible 1
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
INTRODUCTION
Image fusion is the process by which two or more images are combined into
a single image retaining the important features from each of the original images.
The fusion of images is often required for images acquired from different
instrument modalities or capture techniques of the same scene or objects.
Important applications of the fusion of images include medical imaging,
microscopic imaging, remote sensing, computer vision, and robotics. Fusion
techniques include the simplest method of pixel averaging to more complicated
methods such as principal component analysis and wavelet transform fusion.
Several approaches to image fusion can be distinguished, depending on whether
the images are fused in the spatial domain or they are transformed into another
domain, and their transforms fused.
With the development of new imaging sensors arises the need of a
meaningful combination of all employed imaging sources. The actual fusion
process can take place at different levels of information representation, a generic
categorization is to consider the different levels as, sorted in ascending order of
abstraction: signal, pixel, feature and symbolic level. This focuses on the so-called
pixel level fusion process, where a composite image has to be built of several input
images. To date, the result of pixel level image fusion is considered primarily to be
presented to the human observer, especially in image sequence fusion (where the
input data consists of image sequences). A possible application is the fusion of
forward looking infrared (FLIR) and low light visible images (LLTV) obtained by an
airborne sensor platform to aid a pilot navigate in poor weather conditions or
darkness. In pixel-level image fusion, some generic requirements can be imposed
on the fusion result. The fusion process should preserve all relevant information of
the input imagery in the composite image (pattern conservation) The fusion
scheme should not introduce any artifacts or inconsistencies which would distract
the human observer or following processing stages .The fusion process should be
shift and rotational invariant, i.e. the fusion result should not depend on the
location or orientation of an object the input imagery .In case of image sequence
fusion arises the additional problem of temporal stability and consistency of the
fused image sequence. The human visual system is primarily sensitive to moving
Fig. 1 Block Diagram Of Basic Image Fusion Process
7
AIM OF THE PROJECT
2.1 NEW IMAGE FUSION ALGORITHM
The paper adopts the multiresolution analysis discrete wavelet frame
transform and fuzzy region feature fusion scheme to implement the selection of
source image wavelet coefficients. Fig.1 is the framework of the proposed image
fusion algorithm. The first step is to choose an image as object image that can
reflect the object and background clearer than the other image. The second step is
to decompose the source image into multiresolution representation. There are low
frequency band at each level during the next level decomposition. The low
frequency bands of the object image are segmented into region images. The third
step is defining the attributes of the regions by some region features, such as the
mean of gray level in a region. In this case, each pixel point has its membership
value. Then using certain attribute region fusion scheme combining with the
membership value of each pixel, the multiresolution representation of the fusion
result is achieved using defuzzification process. The final step is to do inverse
discrete wavelet frame transform, and the final fusion result is obtained.
8
The fusion of images is the process of combining two or more images into a
single image retaining important features from each. Fusion is an important
technique within many disparate fields such as remote sensing, robotics and
medical applications. Wavelet based fusion techniques have been reasonably
effective in combining perceptually important image features. Shift invariance of
the wavelet transform is important in ensuring robust sub band fusion. Therefore
the novel application of the shift invariant and directionally selective Dual Tree
Complex Wavelet Transform (DT-CWT) to image fusion is now introduced. This
novel technique provides improved qualitative and quantitative results compared to
previous wavelet fusion method.
9
The goals for this Project have been the following.
One goal has been to compile an introduction to the subject of Image
Fusion. There exist a number of studies on various algorithms, but complete
treatments on a technical level are not as common. Material from papers, journals,
and conference proceedings are used that best describe the various parts.
Another goal has been to search for algorithms that can be used to
implement for the image fusion for various applications.
A third goal is to evaluate their performance of with different image quality
metrics. These properties were chosen because they have the greatest impact on
the detection of Image fusion algorithms
A final goal has been to design and implement the Wavelet based fuzzy and
Neural approaches using matlab.
2.2 SCOPE OF THE PROJECT
2.2.1 DWT versus DT-CWT
Figures 3(a) and 3(b) show a pair of multifocus test images that were fused
for a closer comparison of the DWT and DT-CWT methods. Figures 3(d) and 3(e)
show the results of a simple MS method using the DWT and DT-CWT,
respectively. These results are clearly superior to the simple pixel averaging result
shown in 3(c). They both retain a perceptually acceptable combination of the two
“in focus” areas from each input image. An edge fusion result is also shown for
comparison (figure 3(f)) [8]. Upon closer inspection however, there are residual
ringing artefacts found in the DWT fused image not found within the DT-CWT
fused image. Using more sophisticated coefficient fusion rules (such as WBV or
WA) the DWT and DT-CWT results were much more difficult to distinguish.
However, the above comparison when using a simple MS method reflects the
ability of the DT-CWT to retain edge details without ringing.
10
Figure 2.1: (a) First image of the multifocus test set. (b) Second image of the
multi focus test set. (c) Fused image using average pixel values. (d) Fused
image using DWT with an MS fuse rule. (e) Fused image using DT-CWT with
an MS fuse rule. (f) Fused image using multiscale edge fusion
(point representations).
11
2.2.2 Quantitative Comparisons
Often the perceptual quality of the resulting fused image is of prime
importance. In these circumstances comparisons of quantitative quality can often
be misleading or meaningless. However, a few authors [1, 7, 10] have attempted
to generate such measures for applications where their meaning is clearer.
Figures 3(a) and 3(b) reflect such an application: fusion of two images of differing
focus to produce an image of maximum focus. Firstly, a “ground truth” image
needs to be created that can be quantitatively compared to the fusion result
images. This is produced using a simple cut-and-paste technique, physically taking
the “in focus” areas from each image and combining them. The quantitative
measure used to compare the cut-and-paste image to each fused image was
taken from [1]
Figure 2.2: (a) First image (MR) of the medical test set. (b) Second image (CT) of the medical test set. (c) Fused image using average pixel values. (d) Fused image using DWT with an MS fuse rule. (e) Fused image using DT-CWT with an MS fuse rule. (f) Fused image using multiscale edge fusion (point representations).
12
where Igt is the cut-and-paste “ground truth” image, ___ is the fused image and is
the size of the image. Lower values of _ indicate greater similarity between the
images___ and ___ and therefore more successful fusion in terms of quantitatively
measurable similarity. Table 1 shows the results for the various methods used.
The average pixel value method gives a baseline result. The PCA method gave an
equivalent but a slightly worse result. These methods have poor results relatively
to the others. This was expected as they have no scale selectivity. Results were
obtained for the DWT methods using all the bio-orthogonal wavelets available
within the Matlab (5.0) Wavelet Toolbox. Similarly, results were obtained for the
DT-CWT methods using all the shift invariant wavelet filters described in [3].
Results were also calculated for the SIDWT using the Haar wavelet and the
bior2.2 Daubechies wavelet. The table 1 shows the best results for all filters for
each method. For all filters, the DWT results were worse than their DT-CWT
equivalents. Similarly, all the DWT results were worse than their SIDWT
equivalents. This demonstrates the importance of shift invariance in wavelet
transform fusion. The DT-CWT results were also better than the equivalent results
using the SIDWT. This indicates the improvement gained from the added
directional selectivity of the DT-CWT over the SIDWT. The WBV and WA methods
performed better than MS with equivalent transforms as expected, with WBV
performing best for both cases. All of the wavelet transform results were
decomposed to four levels. In addition, the residual low pass images were fused
using simple averaging and the window for the WA and WBV methods were all set
to 3_3.
13
Table 2.1: Quantitative results for various fusion methods.
2.3 EFFECT OF WAVELET FILTER CHOICE FOR DWT AND DT-CWT
BASED FUSION
There are many different choices of filters to affect the DWT transform. In
order not to introduce phase distortions, using filters having a linear phase
response is a sensible choice. To retain a perfect reconstruction property, this
necessitates the use of biorthogonal filters. MS fusion results were compared for
all the images in figures 3 and 4 using all the biorthogonal filters included in the
Mat lab (5.0) Wavelet Toolbox. Likewise there are also many different choices of
filters to affect the DT-CWT transform. MS fusion results were compared for all the
same three image pairs using all the specially designed filters given in [3].
Qualitatively all the DWT results gave more ringing artifacts than the equivalent
DTCWT results. Different choices of DWT filters gave ringing artifacts at different
image locations and scales. The choice of filters for the DT-CWT did not seem to
alter or move the ringing artifacts found within the fused images. The perceived
higher quality of the DT-CWT fusion results compared to the DWT fusion results
was also reflected by a quantitative comparison.
14
WAVELET TRANSFORM OVERVIEW
3.1 WAVELET TRANSFORM
Wavelets are mathematical functions defined over a finite interval and
having an average value of zero that transform data into different frequency
components, representing each component with a resolution matched to its scale.
The basic idea of the wavelet transform is to represent any arbitrary
function as a superposition of a set of such wavelets or basis functions. These
basis functions or baby wavelets are obtained from a single prototype wavelet
called the mother wavelet, by dilations or contractions (scaling) and translations
(shifts). They have advantages over traditional Fourier methods in analyzing
physical situations where the signal contains discontinuities and sharp spikes.
Many new wavelet applications such as image compression, turbulence, human
vision, radar, and earthquake prediction are developed in recent years. In wavelet
transform the basis functions are wavelets. Wavelets tend to be irregular and
symmetric. All wavelet functions, w(2kt - m), are derived from a single mother
wavelet, w(t). This wavelet is a small wave or pulse like the one shown in Fig. 3.2.
Fig. 3.1 Mother wavelet w(t)
Normally it starts at time t = 0 and ends at t = T. The shifted wavelet w(t - m)
starts at t = m and ends at t = m + T. The scaled wavelets w(2kt) start at t = 0 and
end at t = T/2k. Their graphs are w(t) compressed by the factor of 2k as shown in
Fig. 3.3. For example, when k = 1, the wavelet is shown in Fig 3.3 (a). If k = 2 and
3, they are shown in (b) and (c), respectively.
15
(a)w(2t) (b)w(4t) (c)w(8t)
Fig. 3.2 Scaled wavelets
The wavelets are called orthogonal when their inner products are zero. The
smaller the scaling factor is, the wider the wavelet is. Wide wavelets are
comparable to low-frequency sinusoids and narrow wavelets are comparable to
high-frequency sinusoids.
3.1.1 Scaling
Wavelet analysis produces a time-scale view of a signal. Scaling a wavelet
simply means stretching (or compressing) it. The scale factor is used to express
the compression of wavelets and often denoted by the letter a. The smaller the
scale factor, the more “compressed” the wavelet. The scale is inversely related to
the frequency of the signal in wavelet analysis.
3.1.2 Shifting
Shifting a wavelet simply means delaying (or hastening) its onset.
Mathematically, delaying a function f(t) by k is represented by: f(t-k) and the
schematic is shown in fig. 3.4.
(a) Wavelet function (t) (b) Shifted wavelet function (t-k)
Fig. 3.3 Shifted wavelets
16
3.1.3 Scale and Frequency
The higher scales correspond to the most “stretched” wavelets. The more
stretched the wavelet, the longer the portion of the signal with which it is being
compared, and thus the coarser the signal features being measured by the
wavelet coefficients. The relation between the scale and the frequency is shown in
Fig. 3.5.
Low scale High scale
Fig. 3.4 Scale and frequency
Thus, there is a correspondence between wavelet scales and frequency as
revealed by wavelet analysis:
•Low scale a Compressed wavelet Rapidly changing details High