Fourier Transforms - Home - AVN training siteavntraining.hartrao.ac.za/images/MBietenholz_Fourier.pdf · 2016-03-07 · Why is the Fourier Transform Important? • Mainly because

Fourier Transforms

Michael Bietenholz

Acknowledgements

• I have taken much of this presentation from J. J. Condon and S. M. Ransom's "Essential radio Astronomy", course available online at: http://www.cv.nrao.edu/course/astr534/ERA.shtml

What is the Fourier Transform?

• Wikipedia: a Fourier Transform decomposes a function of time into the frequencies that make it up

• In this case, the Fourier transform transforms between the time and the frequency domains

• Easiest to think of in terms of sound: • You don't feel your ear getting hit 220 times a second, you just

hear a single tone. The air molecules hitting your ear 220 times a second is the time-domain representation while the single tone you hear is the frequency-domain representation of the same signal.

• Note that time and frequency have reciprocal units, e.g. unit oftime = 1 sec, unit of frequency = 1 sec-1 = 1 Hz

• Fourier transform is a transform to a reciprocal space, so will always transform from units of X to units of X-1

What Does the Fourier Transform Do?

• What does the Fourier Transform do? Given a smoothie, it finds the recipe.

• How? Run the smoothie through filters to extract each ingredient. • Why? Recipes are easier to analyze, compare, and modify than

the smoothie itself. • How do we get the smoothie back? Blend the ingredients

again.

credit: Kalid

Azad

What is the Fourier Transform?

• Fourier transform is– reversible– linear

• For any function f(x) (which in astronomy is usually real-valued, but f(x) may be complex), the Fourier transform can be denoted F(s), where the product of xand s is dimensionless. Often x is a measure of time t(i.e., the time-domain signal) and so s corresponds to inverse time, or frequency (i.e., the frequency-domainsignal).

Who was Joseph Fourier?

• French mathematician and physicist

• Son of a tailor, orphaned at age 9.• Educated in the Benedictine order

of monks of the Convent of St. Mark.

• He was a strong supporter of the French Revolution, and was imprisoned briefly.

• He accompanied Napoleon Bonaparte on his Egyptian Expedition.

• He developed the Fourier transform while working on ""Théorie analytique de la chaleur" ("On the Propagation of Heat in Solid Bodies" ))

Born: 21 March 1768 in Auxerre, Bourgogne, FranceDied: 16 May 1830 in Paris, France

Why is the Fourier Transform Important?

• Mainly because it often transforms a complicated problem in one domain to a much simpler problem in another domain.

• Example: filtering audio, i.e., say we want to keep only frequencies between 400 and 500 Hz. In the timedomain, where our signal is some function of time, f(t), such filtering is complicated. In the Fourier-transform domain, i.e. in the frequency-domain, this filtering is easy!

• In interferometry because an interferometer by its nature takes measurements in the Fourier-transform plane, in other words in a plane which is the Fourier transform of the image

Expansion of a Function in a Series

• Taylor-series expansion of a functionExample (Taylor Series)

constant

first-orderterm

second-orderterm …

Fourier Series

Fourier series make use of the orthogonalityrelationships of the sine and cosine functions

Fourier showed that any continuous function f(x), can be expressed as a Fourier series

What is a Fourier Transform?

Example: Square Wave

•• Animation of a square waveAnimation of a square wave

•• As more and more Fourier terms or sine waves are As more and more Fourier terms or sine waves are added, the shape more and more closely approaches added, the shape more and more closely approaches a square wavea square wave

Credit:Dr. Dan Russell, Grad. Prog. Acoustics, Penn State

Applications of the Fourier Transform

• Analysis of differential equations (heat conduction, wave propagation...)

• Spectroscopy • Quantum mechanics (wave functions in position and

momentum space)• Antenna design

– Seismic arrays, side scan sonar, GPS– Signal processing– 1D: speech analysis, enhancement …– 2D: image restoration, enhancement …

• Interferometry

Not Just Time and Frequency

• The Fourier transform is equally applicable in other areas to transform between two different representations

• It can easily be generalized to more than one dimension. For example: two dimensions, so it could be used for two dimensional functions, such as an image which can be represented as brightness(x,y)

• In interferometry it is mostly used to transform between the spatial domain – an image (of the sky) –to the spatial frequency domain.

The Fourier Transform

• The Fourier transform is a generalization of the complex Fourier series in the limit

• Fourier analysis = frequency domain analysis – Low frequency: sin(nx), cos(nx) with a small n– High frequency: sin(nx), cos(nx) with a large n

• Note that sine and cosine waves are infinitely long –this is a shortcoming of Fourier analysis (wavelet analysis overcomes this limitation)

One-Dimensional Fourier Transform

• The one-dimensional Fourier transform and its inverse• Fourier transform (continuous case)

• Inverse Fourier transform:

• Alternative definitions of the Fourier transform are based on angular frequency ( =2πν), have different normalizations, or the opposite sign convention in the complex exponential. Since Fourier transformation is reversible, the symmetric symbol is often used to mean "is the Fourier transform of"; e.g., F(s) f(x).

Where i = √-1 in both cases

Complex Exponential

• The heart of the transform is the complex exponential.• A complex exponential is simply a complex number where both

the real and imaginary parts are sinusoids. The exact relation is called Euler's formula

Where i = √-1

• Complex exponentials are much easier to manipulate than trigonometric functions, and they provide a compact notation for dealing with sinusoids of arbitrary phase, which form the basis of the Fourier transform.

credit: Kalid

Azad

Complex Exponential

• Complex exponentials (or sines and cosines) are periodic functions

• Orthogonal and complete set• the Fourier transform can represent any piecewise continuous

function with an error which approaches 0 as N →∞• There exist other complete and orthogonal sets of periodic

functions; for example, Walsh functions (square waves) which are useful for digital electronics.

• Why do we always encounter complex exponentials when solving physical problems? Why are monochromatic waves sinusoidal, and not periodic trains of square waves or triangular waves? The reason is that the derivatives of complex exponentials are just rescaled complex exponentials. This property of complex exponentials makes the Fourier transform uniquely useful in fields ranging from radio propagation to quantum mechanics.

The Discrete Fourier Transform• The continuous Fourier transform converts a time-domain signal

of infinite duration into a continuous spectrum composed of an infinite number of sinusoids.

• Often – especially in astronomy – we deal with signals that are discretely sampled, usually at constant intervals, and of finiteduration or periodic. For such data, only a finite number of sinusoids is needed and the Discrete Fourier Transform (DFT) is appropriate. For almost every Fourier transform theorem or property, there is a related one for the DFT. The DFT of Nuniformly sampled data points xj (where j=0, ... , N−1) and its inverse (Xk) are defined by:

and

The continuous variable s has been replaced by the discrete variable (usually an integer) k.

(Once again, sign and normalization conventions can vary, but this definition is the most common)

Example

• How to make a spike in time out of cosine waves?

• Add four Fourier components each of amplitude 1.0 (and phase 0)

• At time=0, they are all positive (+1.0), and add up to a total amplitude of 4.0

• At other times some are positive and some negative, and add up to a total nearer to 0.

credit: Kalid

Azad

Discrete Fourier Transforms

• The result of the DFT of an N-point input time series is an N-point frequency spectrum, with Fourier frequencies k ranging from −(N/2−1), through the 0-frequency or so-called "DC" component, and up to the highest Fourier frequency N/2−1.

• Each bin number represents the integer number of sinusoidal periods present in the time series.

• The amplitudes and phases represent the amplitudes Ak and phases,Φk ,of those sinusoids.

• Each bin can be described by Xk= Ak e• For real-valued input data, however, the resulting DFT is Hermitian—the

real-part of the spectrum is an even function and the imaginary part is odd, such that X−k=Xk, where the bar represents complex conjugation.

• All of the "negative" Fourier frequencies provide no new information.• Both the k=0 and k=N/2 bins are real-valued, and there is a total of N/2+1

Fourier bins, so the total number of independent pieces of information (i.e. real and complex parts) is N, just as for the input time series. No information is created or destroyed by the DFT.

−iϕk

–

Symmetries

• Other symmetries exist between time- and frequency-domain signals as well:

Frequency DomainTime Domain

real and oddimaginary and odd

imaginary and evenimaginary and even

real and odd (i.e. sine transform)real and odd

real and even (i.e. cosine transform)real and even

oddodd

eveneven

anti-Hermitian (real=odd, imag.=even)imaginaryHermitian (real=even, imag.=odd)real

DFT and FFT

Jean Joseph BaptisteFourier, inventor of

the Fourier transfom

Vinnie "Fast" Fourier, inventor of the Fast

Fourier transfom

DFT and FFT

• Usually the DFT is computed by a very clever (and truly revolutionary) algorithm known as the Fast Fourier Transform or FFT. The FFT was discovered by Gauss in 1805 and re-discovered many times since, but most people attribute its modern incarnation to James W. Cooley and John W. Tukey ("An algorithm for the machine calculation of complex Fourier series," Math. Comput. 19, 297–301) in 1965.

• The key advantage of the FFT over the DFT is that the operational complexity decreases from O(N2) for a DFT to O(N log2(N)) for the FFT.

• Some versions of the FFT are restricted to N = 2,4,8, ... or Nbeing the product of small primes. However, modern implementations of the FFT (such as FFTW) allow O(N log2(N)) complexity for any value of N.

• FFT requires uniformly-spaced points

Fourier Transforms and Sampling

• For a DFT to represent a function accurately, the original function must be sampled at a sufficiently high rate. The appropriate rate for a uniformly sampled time series is determined by the Nyquist-Shannon Theorem or Sampling Theorem:– any continuous baseband signal (signal extending down to

zero frequency) may be identically reconstructed if the signal is bandwidth limited and the sampling frequency is at leasttwice the bandwidth of the signal (i.e. the highest frequency ofa baseband signal).

• That critical sampling rate is 1/Δt, where Δt is the time between successive samples, and is known as the Nyquist rate. It is a property of the time-domain signal based on its frequency content. Somewhat confusingly, if a time-domain signal is sampled uniformly, then the frequency corresponding to one-half the Nyquist rate is called the Nyquist frequency.

Nyquist sampling

• describes the high frequency ( cut-off of the system doing the sampling, and is therefore a property of that system. Any frequencies present in the original signal which are at higher frequencies than the Nyquist frequency will be aliased to other lower frequencies in the sampled band. If the signal was band-limited and then sampled at the Nyquist rate, in accordance to the Sampling Theorem, no aliasing will occur.

• In a DFT, where there are N samples spanning a total time T=NΔt, the frequency resolution is 1/T. Each Fourier bin number k represents exactly k sinusoidal oscillations in the original data xj, and therefore a frequency =k/ΔT in Hz. The Nyquistfrequency corresponds to bin k= N 2T=T (2 T)=NT (2T)=N/2.

• If the signal is not bandwidth limited and higher-frequency components [with k N 2 or N (2T) Hz] exist, those frequencies will show up in the DFT aliased back into lower frequencies fa=N T− ,assuming that N (2T) < N= T. Such aliasing can be avoided by filtering the input data to ensure that it is properly band-limited.

Symmetry

• Even/odd– if g(x) = g(−x), then G(k) = G(−k)– if g(x) = −g(−x), then G(k) = −G(−k)

• Conjugate symmetry– if g(x) is purely real and even, then G(k) is purely real.

• if g(x) is purely real and odd, then G(k) is purely imaginary.

• if g(x) is purely imaginary and even, then G(k) is purely imaginary.

• if g(x) is purely imaginary and odd, then G(k) is purely real.

g(x) = time-domain function;

G(k) = its Fourier transform (frequency domain

Power Spectrum

• A useful quantity in astronomy is the power spectrum

• The power spectrum contains no phase information from the original function.

• Rayleigh's Theorem (sometimes called Plancherel'sTheorem and related to Parseval's Theorem for Fourier series) shows that the integral of the power spectrum equals the integral of the squared modulus of the function (i.e. the energies in the frequency and time domains are equal):

Basic Transforms

I

Basic Transforms

II

Fourier Theorems I

• Addition Theorem: The Fourier transform of the addition of two functions f(x) and g(x) is the addition of their Fourier transforms F(s) and G(s) . This basic theorem results from the linearity of the Fourier transform. A special case of the addition theorem states that if a is a constant, then af(x) aF(s)

• Shift Theorem: a function f(x) shifted along the x-axis by a to become f(x-a) has the Fourier transform e-2πiaF(s). The magnitude of the transform is the same, only the phases change.

f(x) + g(x) F(s) + G(s)

Fourier Theorems II

• Similarity Theorem: For a function f(x) with a Fourier tranformF(s), if the x-axis is scaled by a constant a so that we have f(ax), the Fourier transform becomes . In other words, a "wide" function in the time-domain is a "narrow" function in the frequency-domain. This is the basis of the uncertainty principle in quantum mechanics and the diffraction limits of radio telescopes.

• Modulation Theorem: The Fourier transform of a function f(x) multiplied by is . This theorem is very important in radio astronomy as it describes howsignals can be "mixed" to different intermediate frequencies (i.e. IFs).

Fourier Theorems III

• Derivative Theorem: The Fourier transform of the derivative of a function f(x), f'(x), is i2πsF(s)

Convolution & Cross-Correlation

• Convolution shows up in many aspects of astronomy, most notably in the point-source response of an imaging system and in interpolation.

• We will represent convolution, by ӿ (the symbol is also frequently used)

• multiplies one function, f, by the time-reversed function g, shiftedby some amount x, and integrates from -∞ to +∞.

• The convolution, h(x), of the functions f(x) and g(x) and is a linear function of f and g defined by:

Convolution Example

• Notice how the delta-function part at the right of the top "Function" produces an image of the 'Kernel'

• For a time series, that kernel defines the impulse response of the system

• For an imaging system, the kernel defines the point-spread function

Convolution Theroem

• The convolution theorem is extremely powerful and states that the Fourier transform of the convolution of two functions is the product of their individual Fourier transforms:

• This theorem simplifies many problems: convolution is usually much more computationally intensive than multiplication, so by Fourier transforming first we can replace a complex operation by a simpler one

Cross-Correlation

• Cross-correlation is a very similar operation to convolution, except that the "kernel" is not time-reversed during the operation.

• Cross-correlation is used extensively in interferometry and aperture synthesis imaging,

• also used to perform optimal "matched-filtering" of data to find and identify weak signals.

• We represent cross-correlation is represented by the pentagram symbol * and is defined by:

• In this case, unlike for convolution, f(x)* g(x) ≠ g(x)* f(x)

Autocorrelation

• Autocorrelation is a special case of cross-correlation with . The related autocorrelation theorem is also known as the Wiener-Khinchin theorem and states:

• In words, the Fourier transform of an autocorrelation function is the power spectrum, or equivalently, the autocorrelation is the inverse Fourier transform of the power spectrum.

• Many radio-astronomy instruments compute power spectra using autocorrelations and this theorem.

Auto-Correlation

• One important thing to remember about convolution and correlation using DFTs is that they are cyclic with a period corresponding to the length of the longest component of the convolution or correlation.

• Unless this periodicity is taken into account (usually by zero-padding one of the input functions), the convolution or correlation will wrap around the ends and possibly "contaminate" the resulting function.

One and Two Dimensional Fourier Transforms

• The one-dimensional Fourier transform and its inverse– Fourier transform (continuous case)

– Inverse Fourier transform:

• The two-dimensional Fourier transform and its inverse– Fourier transform (continuous case)

– Inverse Fourier transform:

dueuFxf uxj 2)()(

dydxeyxfvuF vyuxj )(2),(),(

1 where)()( 2

jdxexfuF uxj

dvduevuFyxf vyuxj )(2),(),(

Note the switch in notation from i, used earlier to j on this slide

Fourier Transform of a 2-Dimensional Image

• Each image has three Fourier components

• The center pixel is the offset or DC term, while the other two encode the sinusoidal pattern

• The spacing of the sinusoidal pattern corresponds to the radius of the bright points from the center

• The direction of the sinusoidal pattern corresponds to the position angle of the bright points

Images from Steven Lehar, Boston University

Fourier Transform of a 2-Dimensional Image

• The bottom pair of images is the sum of the two above – the different Fourier components combine additivitely

• The brightness (left) and the Fourier images (right) are completely interchangeable – they contain exactly the same information.

Images from Steven Lehar, Boston University

Fourier Transform of an Image

Original image

Fourier Transform of just Amplitude

Fourier transform: amplitude (log)

Fourier transform: phase

Image credit: Rob Fisher, Univ. Edinburgh