1 The Fast Fourier Transform (and DCT too…) Nimrod Peleg Oct. 2002.

1

The Fast Fourier Transform(and DCT too…)

Nimrod Peleg

Oct. 2002

2

Outline • Introduce Fourier series and transforms• Introduce Discrete Time Fourier Transforms,

(DTFT)• Introduce Discrete Fourier Transforms (DFT)• Consider operational complexity of DFT• Deduce a radix-2 FFT algorithm• Consider some implementation issues of FFTs

with DSPs• Introduce the sliding FFT (SFFT) algorithm

3

The Frequency Domain• The frequency domain does not carry any

information that is not in the time domain.• The power in the frequency domain is that it is simply

another way of looking at signal information. • Any operation or inspection done in one domain is

equally applicable to the other domain, except that usually one domain makes a particular operation or inspection much easier than in the other domain.

• frequency domain information is extremely important and useful in signal processing.

4

3 Basic Representations for FT

• 1. An Exponential Form

• 2. A Combined Trigonometric Form

• A Simple Trigonometric Form

5

The Fourier Series: Exponential Form

Periodic signal expressed as infinite sum of sinusoids.

dte)t(xT

1c

where,ec)t(x

p

0

0

T

tjkp

pk

k

tjkkp

Ck’s are frequency domain amplitude and phase representation For the given value xp(t) (a square value), the sum of the first four terms of trigonometric Fourier series are: xp(t) 1.0 + sin(t) + C2 sin(3t) + C3sin(5t)

Complex Numbers !

6

The Combined Trigonometric Form

• Periodic signal: xp (t) = xp(t+T) for all t

and cycle time (period) is: Tf

1 2

0 0

f0 is the fundamental frequency in Hz w0 is the fundamental frequency in radians:

xp(t) can be expressed as an infinite sum of orthogonal functions. When these functions are the cosine and sine, the sum is called the Fourier Series. The frequency of each of the sinusoidal functions inthe Fourier series is an integer multiple of the fundamental frequency.

Basic frequency + Harmonies

2 f

7

Fourier Series Coefficients

• Each individual term of the series, ,is the frequency domain representation and is generally complex (frequency and phase), but the sum is real.

• The second common form is the combined trigonometric form:

Ckjk te 0

x C C k t

C

C

p t kk

k

kk

k

( ) sin( )

tanIm( )

Re( )

01

1

2 0

Again: Ck are Complex Numbers !

8

The Trigonometric Form

All three forms are identical and are related using Euler’s identity:

0 0

0

0

( ) 0

=1

0 ( ) ( )

( )

( )

= + ( )+ ( )

1= = Average value of

= ( )

= in(

DC

)

p

p

p

p t k k

k

p t p tp T

k p t

T

k p t

T

x A A Cos k t B Sin k t

A dt xT

A Cos k t dt

B S k t dt

x

x

x

e Cos jSinj Thus, the coefficients of the different forms are related by:

2 0 0

1 1

C A jB C A

B

A

C

C

k k k

kk

k

k

k

; =

tan tanIm( )

Re( )

j -j

j -j

Reminder:

e eCos =

2

e -e=

2jSin

9

The Fourier Transform 1/3

The Fourier series is only valid for periodic signals.

For non-periodic signals, the Fourier transform is used.

Most natural signals are not periodic (speech).We treat it as a periodic waveform with an infinite period. If we assume that TP tends towards infinity, then we can produce equations (“model”) for non-periodic signals.

If Tp tends towards infinity, then w0 tends towards 0. Because of this, we can replace w0 with dw, :and it leads us to

limt pT

d

1

2

12p

pfT

10

The Fourier Transform 2/3

Increase TP = Period Increases : No Repetition:

Discrete coefficients Ck become continuous:

Discrete frequency variable becomes continuous:

Cd

x e dttj t

( ) ( )

z2

1

2 2T

d

p

k 0

C Ck ( )

( )( )2 j tt

Cx e dt

d

11

The Fourier Transform 3/3

We define: ( )2 ( ) ( )

CX x t

d

F{

X x e dt x t x e dtj t j t

( ) ( ) ( )( )

z z 1

2

12

Signal Representation by Delta Function Instead of a continuous signal we have a “collection of samples”:

This is equivalent to sampling the signal with one DeltaFunction each time, moving it along X-axis, and summing all the results:

x xs t t t nTs( ) ( ) ( )

Note that the Delta is “1” only If its index is zero !

13

Discrete Time Fourier Transform 1/3

• Consider a sampled version, xs(t) , of a continuous signal, x(t) : x xs t t t nTs( ) ( ) ( )

Ts is the sample period. We wish to take the Fourier transform of this sampled signal. Using the definition of Fourier transform of xs(t) and some mathematical properties of it we get:

x x es nTsj nTs

n( ) ( )

Replace continuous time t with (nTs) Continuous x(t) becomes discrete x(n) Sum rather than integrate all discrete samples

14

Discrete Time Fourier Transform 2/3

Fourier Transform

Discrete Time Fourier Transformdtetxx

tj

)()(

n

j ne)n(x)(x

de)(x2

1)t(x

tj

de)(x2

1)n(x

)n(jInverse Fourier

Transform

Inverse Discrete

Time Fourier Transform

Limits of integration need not go beyond ± because the spectrum repeats itself outside ± (every 2):

Keep integration because is continuous: means that is periodic every Ts

X X( ) ( ) 2

X ( ) Ts

15

Discrete Time Fourier Transform 3/3

• Now we have a transform from the time domain to the frequency domain that is discrete, but ...

DTFT is not applicable to DSP because it

requires an infinite number of samples and

the frequency domain representation is a

continuous function – impossible to represent

exactly in digital hardware.

16

1st result: Nyquist Sampling Rate 1/2

• The Spectrum of a sampled signal is periodic, with 2*Pi Period: ( ) ( 2 )X X

Easy to see: ( 2 ) 2( 2 ) ( ) ( )

( ) ( )

jn jn j n

n n

jn

n

X x n e x n e e

x n e X

2 cos(2 ) sin(2 ) 1j ne n j n

17

1st result: Nyquist Sampling Rate 2/2

• For maximum frequency wH :

|

:

2 =2

H

H

H s

s H

Ts

Ts

Ts BUT Ts

Ts

18

Practical DTFT

Take only N time domain samples

Sample the frequency domain, i.e. only evaluate x() at N

discrete points. The equal spacing between points is = 2/N

n

j n

enxx )()(

1

0

)()(N

n

j n

enxx

xk

Nx n e k Nj

n

N kn N

( ) ( ) , , ,...,/2

0 1 2 12

0

1

19

The DFT

The result is called Discrete Fourier Transform (DFT):

Since the only variable in is k , the DTFT is written:2k N/

x k x n e k Nj

n

N kn N

( ) ( ) , , ,...,/

2

0

1

0 1 2 1

Using the shorthand notation: (Twiddle Factor)W eNj N 2 /

X k x n W and x nN

X k WN Nkn

n

N

N Nkn

k

N

( ) ( ) ( ) ( )

0

1

0

11

20

Usage of DFT

• The DFT pair allows us to move between the time and frequency domains while using the DSP.

• The time domain sequence x[n] is discrete and has spacing Ts, while the frequency domain sequence X[k] is discrete and has spacing 1/NT [Hz].

21

DFT RelationshipsTime Domain Frequency Domain

|x(k)|

0

0

1 2 N/2 N-2 N-1

N

FsN

F2 s

2

FsN

F2 sN

Fs

N Samples

k

f

N Samples

0

0

Ts

1

2Ts

2

3Ts

3 N-1

(N-1)Ts

X(n)

t

n

22

Practical Considerations

1000-point DFT requres 10002 = 106 complex multiplications And all of these need to be summed….

Standard DFT:

An example of an 8 point DFT:

Writing this out for each value of n :

0k7W)0(x Each term such as requires 8 multiplications

Total number of (complex !) multiplications required: 8 * 8 = 64

1

0

( ) ( ) 0 1N

knN n N

n

X k x k W k N

7,...,1,0k,W)7(x.......W)1(xW)0(x)k(X 7k7

1k7

0k7n

7

70

( ) ( ) 0,1,2,...,7knN N

n

X k x k W k

23

Fast Fourier TransformSymmetry Property

Periodicity Property

THE FAST FOURIER TRANSFORM

Splitting the DFT in two(odd and even)

or

Manipulating the twiddle factor

kN

2/NkN WW

kN

NkN WW

2/N

)2/N

2(j)2

N

2(j2

N WeeW

1

2

N

0r

rk2N

12

N

0r

kN

rk2NN )W).(1r2(xW)W).(r2(x)k(X

1

2

N

0r

k)1r2(N

12

N

0r

rk2NN W).1r2(xW).r2(x)k(X

1

2

N

0r

12

N

0r

rk2N

kN

rk2Nn W)1r2(xWW)r2(x)k(X

24

FFT complexity

N/2 Multiplications

For an 8-point FFT, 42 + 42 + 4 = 36 multiplications, saving 64 - 36 = 28 multiplications

For 1000 point FFT, 5002 + 5002 + 500 = 50,500 multiplications, saving 1,000,000 - 50,500 = 945,000 multiplications

12

02/

12

02/ )12()2()(

N

r

rkN

kN

N

r

rkNN WrxWWrxkx

(N/2)2 multiplications (N/2)2 multiplications

25

Time Decimation

Decimate once Called Radix-2 since we divided by 2

Splitting the original series into two is called decimation in time

n = {0, 1, 2, 3, 4, 5, 6, 7}n = { 0, 2, 4, 6 } and { 1, 3, 5, 7 }

Let us take a short series where N = 8

Decimate againn = { 0, 4 } { 2, 6 } { 1, 5 } and { 3, 7 }

The result is a savings of N2 – (N/2)log2N multiplications:1024 point DFT = 1,048,576 multiplications1024 point FFT = 5120 multiplication

Decimation simplifies mathematics but there are more twiddlefactors to calculate, and a practical FFT incorporates these extra factors into the algorithm

26

Simple example: 4-Point FFT Let us consider an example where N=4:

Decimate in time into 2 series: n = {0 , 2} and {1, 3}

We have two twiddle factors. Can we relate them?

Now our FFT becomes:

X x n Wk kn4 4

0

3

( ) ( )

X x r W W x r W

x x W W x x W

k rk

r

k rk

r

k k k

4 20

1

4 20

1

2 4 2

2 2 1

0 2 1 3

( ) ( ) ( )

{ ( ) ( ) } { ( ) ( ) }

W e

W e e W

Nk j

Nk

k j k j k k

2

2

22

24

2

22

X x x W W x x Wkk k k

4 42

4 420 2 1 3( ) { ( ) ( ) } { ( ) ( ) }

27

4-Point FFT Flow DiagramThe 2 DFT’s:for k=0,1,2,3

X x x W W x x Wkk k k

4 42

4 420 2 1 3( ) { ( ) ( ) } { ( ) ( ) }

For k=0 only: X x x W W x x W4 0 40

40

400 2 1 3( ) { ( ) ( ) } { ( ) ( ) }

A ‘flow-diagram’ of it:

x(0)

x(2)

W40

+

x(1)

x(3)

W40

+

W40

+

This is for only 1/4 of thewhole diagram !

28

A Complete Diagram

x(0)

x(1)

x(2)

x(3)

X4(0)

X4(1)

X4(2)

X4(3)

0

2

0

2 3

2

1

0

X x x W W x x W4 0 40

40

400 2 1 3( ) { ( ) ( ) } { ( ) ( ) }

X x x W W x x W4 1 42

41

420 2 1 3( ) { ( ) ( ) } { ( ) ( ) }

X x x W W x x W4 2 40

42

400 2 1 3( ) { ( ) ( ) } { ( ) ( ) }

X x x W W x x W4 3) 42

43

420 2 1 3( { ( ) ( ) } { ( ) ( ) }

Note: W eNk j

Nk

2

W e Wj

44

2

44

401

W e Wj

46

2

46

421

29

The Butterfly

X1

X2x2

WNk

x1 x1X1 = + WNk x2

x1X2 = – WNk x2

A Typical Butterfly

W43 = j

W42 = -1

W41 = -j

W40 = 1

Twiddle Conversions

X0 = (x0 + x2) + W40 (x1+x3)

X1 = (x0 – x2) + W41 (x1–x3)

X2 = (x0 + x2) – W40 (x1+x3)

X3 = (x0 – x2) – W41 (x1–x3)

4 Point FFT Equations

X0

X1

X3

X2

x3

x2

x1

x0

W40

W41

4 Point FFT Butterfly

30

Summary Frequency domain information for a signal is important for processing

Sinusoids can be represented by phasors

Fourier series can be used to represent any periodic signal

Fourier transforms are used to transform signals From time to frequency domain From frequency to time domain

DFT allows transform operations on sampled signals

DFT computations can be sped up by splitting the original series into two or more series

FFT offers considerable savings in computation time

DSPs can implement FFT efficiently

31

Bit-Reversal• If we look at the inputs to the butterfly FFT, we can see

that the inputs are not in the same order as the output. • To perform an FFT quickly, we need a method of

shuffling these input data addresses around to the correct order.

• This can be done either by reversing the order of the bits that make up the address of the data, or by pointer manipulation (bit reversed addition).

• Many DSPs have special addressing modes that allow them to automatically shuffle the data in the background.

32

Bit-Reversal example

• To obtain the output in ascending order the input values must be loaded in the order: {0,2,1,3}

• for 512 or 1024 it is much more complicated...

x(0)

x(1)

x(2)

x(3)

X4(0)

X4(1)

X4(2)

X4(3)

0

2

0

2 3

2

1

0

33

8-point Bit-Reversal• Consider a 3-bit address (8 possible locations).• After starting at zero, we add half of the FFT

length at each address access with carrying from left to right (!)

Start at 0 = 000 =x(0)000+100 = 100 =x(4)100+100 = 010 =x(2)010+100 = 110 =x(6)110+100 = 001 =x(1)001+100 = 101 =x(5) 101+100 = 011 =x(3)011+100 = 111 =x(7)

Note that reversing the order of the address bits gives same result !

34

And what about DCT ???

The rest of the math is quite similar…..Note: at least 4 types of DCT !!!

1 1II 22

0

1 1II 22

0

12

( )(k)= b(k) ( )cos

( )( ) b(k)X (k)cos

if k=0 b(k)=

1 if k=1,...,L-1

N

Nn

N

Nk

n kX x n

N

n kx n

N

DCT Type II*:

* after: A course in Digital Signal Processing, Boaz Porat

35

DCT Type II

DCT Basis Vectors for N=8

Type I Type II Type III Type IV

Most used for compression:JPEG, MPEG etc.

36

DCT Features

• Real transformation

• Reversible transformation

• 2D Transformation exists and separable

• Better than the DFT as a “de-correlator”

• Fast algorithm exists (NlogN Complexity)