From controller to code

From controller/filter to code:the optimal implementation problemThibault Hilaire ([email protected])

From controller/filter to code: the optimal implementation problem Thibault Hilaire ([email protected]) 1/72

Context Linear filters and equivalent realizations

Context andproblematicsContext andproblematics11



Outline

1 Context

2 Linear filters and equivalent realizations



Context

implantation

0100101101010100010111101010

Ciblesalgorithmes TSTargets

Implementationimplantation

0100101101010100010111101010

Ciblesalgorithmes TSFilters/ControllersAlgorithms

• Finite precision implementation (fixed-point arithmetic)• LTI systems• hardware (FPGA, ASIC) or software (DSP, µC)

Methodology for the implementation of embedded algorithms(controllers/filters)



Context


0100101101010100010111101010


implantation

0100101101010100010111101010






Context

implantation

0100101101010100010111101010



0100101101010100010111101010






Context

implantation

0100101101010100010111101010



0100101101010100010111101010






Context

implantation

0100101101010100010111101010



0100101101010100010111101010






Implementation under constraintsEmbedded systems suffer from various constraints:

• Cost (short Time to Market, rapid renewal, etc...)• Power consumption (energy-autonomous systems)• Resources (Limited computing resources (no FPU))

The implementation of a given algorithm is difficult:• Deal with precision problems

• Implementation ⇒ accuracy problems• Fixed-point arithmetic

• Area/power consumption constraints• multiple wordlength paradigm (ASIC and FPGA)

• Time-consuming work

We need tools to transform controller into code that deal withprecision/performance/computational cost tradeoff





















Objectives

The objectives of the presentation is to give an overview of:• the interest and the difficulties of this problem• the parts already (almost) done• the various parts in-progress• some analogies with the optimal polynomial implementationproblem



Finite precision degradations

The implementation ⇒ degradations (performances, characteristics, etc.)

Origins of the degradationsTwo sources of deteriorations are considered:

• Roundoff errors into the computations→ roundoff noise

• Quantization of the coefficients→ parametric errors

The degradations depends on• the algorithmic relationship used to compute output(s) frominput(s)

• the computations themselves (wordlengths, roundoff mode,binary point position, additions order, etc.)






























Linear filters

LTI: Linear system with known and constant coefficients• FIR (Finite Impulse Response) :

H(z) =n∑

i=0

biz−i , ⇒ y(k) =n∑

i=0

biu(k − i)

• IIR (Infinite Impulse Response) :

H(z) =

∑ni=0 biz−i

1 +∑n

i=1 aiz−i , ⇒ y(k) =n∑

i=0

biu(k−i)−n∑

i=1

aiy(k−i)

Filters and signal processing



Equivalent realizationsFor a given LTI controller, it exist various equivalent realizations

• Direct Form I (2n + 1 coefs)

• Direct Form II (transposed or not) (2n + 1 coefs)• State-space (depend on the basis) ((n + 1)2 coefs)• δ-operator δ , q−1

∆• Cascad and/or parallel decompositions• ρDirect Form II transposed (5n + 1 ou 4n + 1 coefs) ρDFIIt

• lattice, ρ-modale, LGC et LCW-structures, sparse state-space,etc.

H(z) =

∑ni=0 biz−i

1 +∑n

i=1 aiz−i

y(k) =n∑

i=0

biu(k−i)−n∑

i=1

aiy(k−i)

q!1 q!1 q!1 q!1

q!1q!1q!1q!1

+

b0 b1 b2 bi bn

!a1!a2!ai!an

u(k) u(k − 1) u(k − n)

y(k − n) y(k − 1)

y(k)




• Direct Form I (2n + 1 coefs)• Direct Form II (transposed or not) (2n + 1 coefs)

• State-space (depend on the basis) ((n + 1)2 coefs)• δ-operator δ , q−1



q!1

q!1

q!1

q!1

+

+

+

+

+

+

+

+

!a1

!a2

!ai

!an

u(k) y(k)e(k)

e(k − 1)

e(k − n)

b0

b1

bi

b2

bn




• Direct Form I (2n + 1 coefs)• Direct Form II (transposed or not) (2n + 1 coefs)• State-space (depend on the basis) ((n + 1)2 coefs)

• δ-operator δ , q−1∆

• Cascad and/or parallel decompositions• ρDirect Form II transposed (5n + 1 ou 4n + 1 coefs) ρDFIIt


X (k + 1) = AX (k) + Bu(k)

y(k) = CX (k) + Du(k)

X (k)→ T .X (k)q!1

+

A

+CB

D

U(k) Y (k)

X(k)X(k + 1)




• Direct Form I (2n + 1 coefs)• Direct Form II (transposed or not) (2n + 1 coefs)• State-space (depend on the basis) ((n + 1)2 coefs)• δ-operator δ , q−1

∆

• Cascad and/or parallel decompositions• ρDirect Form II transposed (5n + 1 ou 4n + 1 coefs) ρDFIIt


q!1+!

!!1

δ[X (k)] = AδX (k) + Bδu(k)y(k) = CδX (k) + Dδu(k)

+ +

U(k) Y (k)

X(k)X(k + 1)!!1

A!

B! C!

D!





∆• Cascad and/or parallel decompositions

• ρDirect Form II transposed (5n + 1 ou 4n + 1 coefs) ρDFIIt


H1 H2 Hi Hm

H

+

H

H1

H2

Hi

Hm







+ !iz!1

!i!!1

i

++ ++ +

!n !i!n!1 !1 !0

!!1n !!1

i+1 !!1i !!1

1

!n !n!1 !i !1

y(k)

u(k)







Each of these realizations uses different coefficients.They are equivalent in infinite precision, but no more in finiteprecision. The finite precision degradation (parametric errors androundoff noises) depends on the realization.



Comparison with polynomials – 1

It is very interesting to compare filter realizations and polynomialevaluation:• The shift operator z!1 • The multiplication by x

• Direct forms • Horner’s method• ρ-operator

%i (z) =n∏

j=i+1

z − γj

∆j

• Newton’s basis

Pi = Πik=1(X − xk)

• ρDFIIt • Horner’s method with Newton’sbasis




It is very interesting to compare filter realizations and polynomialevaluation:• The shift operator z!1 • The multiplication by x• Direct forms • Horner’s method

• ρ-operator

%i (z) =n∏

j=i+1

z − γj

∆j







It is very interesting to compare filter realizations and polynomialevaluation:• The shift operator z!1 • The multiplication by x• Direct forms • Horner’s method• ρ-operator

%i (z) =n∏

j=i+1

z − γj

∆j







It is very interesting to compare filter realizations and polynomialevaluation:• The shift operator z!1 • The multiplication by x• Direct forms • Horner’s method• ρ-operator

%i (z) =n∏

j=i+1

z − γj

∆j







• State-space

X (k + 1) = AX (k) + Bu(k)y(k) = CX (k) + Du(k)

•

X0 =(1 0 . . . 0

)>Xi+1 =

x1 0

. . .. . .1 0

Xi

P =(an an−1 . . . a0

)Xn

• Basis change with any T invertible

Could it be interesting to consider other polynomial evaluationschemes (Estrin, Knuth&Eve, etc.) ?




• State-space

X (k + 1) = T−1ATX (k) + T−1Bu(k)y(k) = CTX (k) + Du(k)

•

X0 =(1 0 . . . 0

)>Xi+1 = T−1

x1 0

. . .. . .1 0

TXi

P =(an an−1 . . . a0

)TXn






• State-space

X (k + 1) = T−1ATX (k) + T−1Bu(k)y(k) = CTX (k) + Du(k)

•

X0 =(1 0 . . . 0

)>Xi+1 = T−1

x1 0

. . .. . .1 0

TXi

P =(an an−1 . . . a0

)TXn





HW or SW implementation

In addition to the choice of the realization, a lot of other options:• Fixed-point evaluation schemes

• order of the operations• fixed-point alignments• quantization modes, etc.

• Software implementation (µC, DSP, etc.):• Parallelization• Ressource allocations

• Hardware implementation (FPGA, ASIC)• Multiple word-length paradigm• Operators optimizations (multiplications by constant, etc.)• Hardware-oriented arithmetic• Operators regroupment

Different levels of optimizations !


























Objectives SIF Criteria Evaluation scheme Methodology

Our approachOur approach22



Outline

3 Objectives

4 SIF

5 Criteria

6 Evaluation scheme

7 Methodology



Objectives

Given a controller and a target, find the optimal implementation.

• Model all the equivalent realizations• SIF (Specialized Implicit Form)

• Model the fixed-point algorithms• depends on evaluation schemes for sum-of-products

• Model the hardware resources• computational units, etc.

• Evaluate the degradation• Find one/some optimal realizationsTradeoff between:

• Performance / Accuracy (multi-criteria)• Computational cost• Area, power consumption• Computation time, reusability, off-line efforts, etc.



From controller to codetransfer function, state-space, etc.

set of equivalent realizations

structure

implementation

realization choice

SIF

langage

code

code generation

Z

fixed-point algorithm (Z*, ...)

abstract syntax treeword-length

arithmetic operators

roundoff modes...

T ,∆, γ, ...



A Unifying FrameworkWe have shown that it was possible to describe all these realizationsin a descriptor framework called Specialized Implicit Form (SIF).

• Macroscopic description of the computations• More general than state-space• Explicit all the operations and their linking J 0 0

−K I 0−L 0 I

T (k + 1)X (k + 1)Y (k)

=

0 M N0 P Q0 R S

T (k)X (k)U(k)

SIF


Realizations




−K I 0−L 0 I

T (k + 1)X (k + 1)Y (k)

=

0 M N0 P Q0 R S

T (k)X (k)U(k)

→ it is able to represent all the graph flows with delay, additionsand constant multiplications

SIF


Realizations




−K I 0−L 0 I

T (k + 1)X (k + 1)Y (k)

=

0 M N0 P Q0 R S

T (k)X (k)U(k)

We denote Z the matrix containing all thecoefficients (and the structure of therealization)

Z ,

−J M NK P QL R S

SIF


Realizations


Different criteriaWe have three kinds of criteria to evaluate the finite-precisiondegradation:

• Performance:• transfer function error

∥∥H − H†∥∥• poles or zeros errors

∥∥|λ| − |λ†|∥∥• weighted errors

• Accuracy:• Signal to Quantization Noise Ratio

• Computational cost:• number of coefficients, number of operations• area, power consumption (FPGA, ASIC)• WCET• complexity, etc.

→ These criteria require to know the exact implementation.So we extend them in a priori criteria (that do not depend onimplementation) and a posteriori criteria.



























→ These criteria require to know the exact implementation.

So we extend them in a priori criteria (that do not depend onimplementation) and a posteriori criteria.












a priori criteria – 1

These criteria come from control:

a priori measures• sensitivity with respect to the coefficients Z

• transfer function sensitivity : ∂H∂Z

• poles or zeros sensitivity : ∂|λk |∂Z

• Roundoff Noise Gainwe suppose here that each quantization error leads to the addition of awhite independent uniform noise→ Gain from these noises to the output


a priorimeasures









a priorimeasures









a priorimeasures



The idea is to approximate how much the transfer function ismodified by the coefficients’ quantization.Denote

(δZ )i ,j =

0 if Zi ,j could be exactly implemented ∈ 2p|p ∈ Z1 else

Then, the transfer function error is approached by the L2-sensitivitymeasure

L2-sensitivity measure

ML2 =

∥∥∥∥∂H∂Z × δZ

∥∥∥∥2

2


a priorimeasures


a priori criteria – 3In a same way, the pole error is approached by a pole-sensitivitymeasured.

pole-sensitivity

Ψ ,n∑

k=1

ωk

∥∥∥∥∂ |λk |∂Z

× δZ

∥∥∥∥2

F

where λk are the poles, and ωk some wieghting parameters (usuallyωk = 1

1−|λk |)

→In closed-loop, Ψ is related to the distance to instability.

These measures have been developed for the SIF:• MIMO case• open and closed-loop cases• analytical expressions

And extended with more accurate fixed-point considerations σ2δH


a priorimeasures



pole-sensitivity

Ψ ,n∑

k=1

ωk

∥∥∥∥∂ |λk |∂Z

× δZ

∥∥∥∥2

F


1−|λk |)





a priorimeasures



pole-sensitivity

Ψ ,n∑

k=1

ωk

∥∥∥∥∂ |λk |∂Z

× δZ

∥∥∥∥2

F


1−|λk |)





a priorimeasures


a posteriori measure

The main measure based on exact implementation is the Signal toQuantization Noise Ratio SQNR :

a posteriori measureIt measures the noise on the output produced by the implementation

• bit-accurate measure• depends on HW/SW

• wordlength• evaluation scheme (order, roundoff mode, etc.)

• an optimization on the word-length can be done

→ For that purpose, we need to generate bit-accurate fixed-pointalgorithms


a posteriorimeasures


Sum-of-products evaluation scheme – 1The only operations needed are sum-of-products:

S =n∑

i=1

ci · xi

where ci are known constants and xi variables (inputs, state orintermediate variables).We have developed technics to transform real sum-of-products intofixed-point algorithms:

• we consider the Πni=1(2i − 1) possible ordered-sum-of-products

(assuming the fixed-point addition is commutative but not associative)

• given the various word-lengths (operators, constants), wepropagate fixed-point rules among the abstract syntax tree

• we consider various scheme (RBM/RAM, no-shifts, etc.)


fixed-pointevaluationschemes


Sum-of-products evaluation scheme – 2S = 1.524·X0+4.89765·X1−42.3246·X2+58.35498·X3+2.857·X4−49.3246·X5+24.3468·X6

• 8-bit coefficients, 16-bit additions• Q6.2 for xi and s

→The roundoff noise is then deduced






Sujet de la These Travaux realises jusqu’ici

Ordered Sum of Products

On fixe ici les representations virgule fixe :

8 bits signes pour les coefficients

Q6, 2 pour les Xi et S .

S

+

+

>> 4

+

>> 1

×

98 X0

×

91 X4

+

+

>> 3

×

78 X1

×

-85 X2

×

-99 X5

+

×

117 X3

>> 1

×

97 X6

Roundoff After MultiplicationRoundoff After Multiplication

→The roundoff noise is then deducedFrom controller/filter to code: the optimal implementation problem Thibault Hilaire ([email protected]) 24/72





Sujet de la These Travaux realises jusqu’ici

Ordered Sum of Products

On fixe ici les representations virgule fixe :

8 bits signes pour les coefficients

Q6, 2 pour les Xi et S .

S

+

+

>> 4

+

×

>> 1

98

X0

×

91 X4

+

+

×

>> 3

78

X1

×

-85 X2

×

-99 X5

+

×

117 X3

×

>> 1

97

X6

Roundoff Before MultiplicationRoundoff Before Multiplication

→The roundoff noise is then deducedFrom controller/filter to code: the optimal implementation problem Thibault Hilaire ([email protected]) 24/72



State-space example

X (k + 1) =

(0.58399 −0.420190.42019 0.1638

)X (k) +

(0.64635−0.23982

)Y (k) =

(0.64635 0.23982

)X (k) + 0.13111U(k)



State-space example

X (k + 1) =

(0.58399 −0.420190.42019 0.1638

)X (k) +

(0.64635−0.23982

)Y (k) =

(0.64635 0.23982

)X (k) + 0.13111U(k)

16-bit input, output, states16-bit coefficients32-bit accumulatorsmaxU = 10

⇒ γU = 11

γADD =

262726

γZ =

15 16 1516 17 1715 17 17

γX =

(1111

)

Roundoff After MultiplicationIntermediate variablesAcc ← (xn(1) ∗ 19136);Acc ← Acc + (xn(2) ∗ −27537) >> 1;Acc ← Acc + (u ∗ 21179);xnp(1)← Acc >> 15;Acc ← (xn(1) ∗ 27537);Acc ← Acc + (xn(2) ∗ 21470) >> 1;Acc ← Acc + (u ∗ −31433) >> 1;xnp(2)← Acc >> 16;OutputsAcc ← (xn(1) ∗ 21179);Acc ← Acc + (xn(2) ∗ 31433) >> 2;Acc ← Acc + (u ∗ 17184) >> 2;y ← Acc >> 15;Permutationsxn ← xnp;



State-space example

X (k + 1) =

(0.58399 −0.420190.42019 0.1638

)X (k) +

(0.64635−0.23982

)Y (k) =

(0.64635 0.23982

)X (k) + 0.13111U(k)

16-bit input, output, states16-bit coefficients32-bit accumulatorsmaxU = 10 ⇒ γU = 11

γADD =

262726

γZ =

15 16 1516 17 1715 17 17

γX =

(1111

)

Roundoff After MultiplicationIntermediate variablesAcc ← (xn(1) ∗ 19136);Acc ← Acc + (xn(2) ∗ −27537) >> 1;Acc ← Acc + (u ∗ 21179);xnp(1)← Acc >> 15;Acc ← (xn(1) ∗ 27537);Acc ← Acc + (xn(2) ∗ 21470) >> 1;Acc ← Acc + (u ∗ −31433) >> 1;xnp(2)← Acc >> 16;OutputsAcc ← (xn(1) ∗ 21179);Acc ← Acc + (xn(2) ∗ 31433) >> 2;Acc ← Acc + (u ∗ 17184) >> 2;y ← Acc >> 15;Permutationsxn ← xnp;



Optimal realization

A 1st possible optimization concerns the realization

Let H be a given controller, we denote RH the set of equivalentrealizations (with H as transfer function).Let J be an a priori criterion (sensibility, RNG, σ2

∆H , ...).

The optimal realization problemThe optimal realization problem consists in finding a realizationthat minimizes J

Ropt = arg minR∈RH

J (R)

RH is too large, so pratically we only considered structuredrealizations (state-space, ρ-forms, etc.)


realizationoptimization


Optimal realization



∆H , ...).



J (R)





Optimal realization



∆H , ...).



J (R)





Example – 1

We consider the following transfer function:

H : z 7→ 38252z3 − 101878z2 + 91135z − 27230z4 − 2.3166z3 + 2.1662z2 − 0.96455z + 0.17565

• Closed-loop L2-sensitivity ML2

• Closed-loop pole-sensitivity : Ψ

• Roundoff Noise Gain (RNG) : G



Example – 2MW

L2 Ψ G TO Nb. op.Z1 1.9046e+7 3.3562e+7 1.186e+6

3.6764e+8

7 + 8×

Z2 3.6427e+5 6.5007e+5 3.6582e+2 1.1387e+5 19 + 24×

Z3 1.5267e+3 1.6689e+4 1.7455e+2 5.4111e+4 19 + 24×Z4 1.6272e+3 2.7425e+3 1.1778e+2 3.6512e+4 19 + 24×Z5 1.9474e+13 1.2294e+13 3.2261e−3 1.7239e+10 19 + 24×Z6 2.8696e+3 4.5371e+3 7.9809e−3 6.0078e+0 19 + 24×

Z1: canonical form



Example – 2MW

L2 Ψ G TO Nb. op.Z1 1.9046e+7 3.3562e+7 1.186e+6 3.6764e+8 7 + 8×Z2 3.6427e+5 6.5007e+5 3.6582e+2

1.1387e+5

19 + 24×

Z3 1.5267e+3 1.6689e+4 1.7455e+2 5.4111e+4 19 + 24×Z4 1.6272e+3 2.7425e+3 1.1778e+2 3.6512e+4 19 + 24×Z5 1.9474e+13 1.2294e+13 3.2261e−3 1.7239e+10 19 + 24×Z6 2.8696e+3 4.5371e+3 7.9809e−3 6.0078e+0 19 + 24×

Z1: canonical formZ2: balanced state-space



Example – 2MW


1.1387e+5

19 + 24×Z3 1.5267e+3 1.6689e+4 1.7455e+2

5.4111e+4

19 + 24×

Z4 1.6272e+3 2.7425e+3 1.1778e+2 3.6512e+4 19 + 24×Z5 1.9474e+13 1.2294e+13 3.2261e−3 1.7239e+10 19 + 24×Z6 2.8696e+3 4.5371e+3 7.9809e−3 6.0078e+0 19 + 24×

Z3: ML2-optimal state-space



Example – 2MW


1.1387e+5

19 + 24×Z3 1.5267e+3 1.6689e+4 1.7455e+2

5.4111e+4

19 + 24×Z4 1.6272e+3 2.7425e+3 1.1778e+2

3.6512e+4

19 + 24×

Z5 1.9474e+13 1.2294e+13 3.2261e−3 1.7239e+10 19 + 24×Z6 2.8696e+3 4.5371e+3 7.9809e−3 6.0078e+0 19 + 24×

Z4: Ψ-optimal state-space



Example – 2MW


1.1387e+5

19 + 24×Z3 1.5267e+3 1.6689e+4 1.7455e+2

5.4111e+4

19 + 24×Z4 1.6272e+3 2.7425e+3 1.1778e+2

3.6512e+4

19 + 24×Z5 1.9474e+13 1.2294e+13 3.2261e−3

1.7239e+10

19 + 24×

Z6 2.8696e+3 4.5371e+3 7.9809e−3 6.0078e+0 19 + 24×

Z5: G -optimal state-space



Example – 2MW

L2 Ψ G TO Nb. op.Z1 1.9046e+7 3.3562e+7 1.186e+6 3.6764e+8 7 + 8×Z2 3.6427e+5 6.5007e+5 3.6582e+2 1.1387e+5 19 + 24×Z3 1.5267e+3 1.6689e+4 1.7455e+2 5.4111e+4 19 + 24×Z4 1.6272e+3 2.7425e+3 1.1778e+2 3.6512e+4 19 + 24×Z5 1.9474e+13 1.2294e+13 3.2261e−3 1.7239e+10 19 + 24×Z6 2.8696e+3 4.5371e+3 7.9809e−3 6.0078e+0 19 + 24×

Tradeoff measure: we are looking for a good enough realization:

TO(Z ) ,MW

L2 (Z )

MW optL2

+Ψ(Z )

Ψopt+

G (Z )

G opt

Z6: TO-optimal state-space



Example – 3

MWL2 Ψ G TO Nb. op.

Z7 1.5342e−2 8.1051e−2 2.8082e−8 4.5466e+0 11 + 12×Z8 1.5341e−2 8.089e−2 4.217e−8 4.8783e+0 11 + 16×Z9 1.1388e−1 2.8203e−2 3.7783e−6 9.8937e+1 11 + 16×Z10 1.5342e−2 8.0015e−2 4.1742e−8 4.8371e+0 11 + 16×Z11 1.6065e−2 3.8802e−2 4.7451e−8 3.5597e+0 11 + 16×

Z7: γ =(1111

)> : δ-Direct Form IIZ8: MW

L2 -optimal ρDFIItZ9: Ψ-optimal ρDFIItZ10: G -optimal ρDFIItZ11: TO-optimal ρDFIIt



Word-lengths optimization

A 2nd possible optimization concerns the hardware implementation

On FPGA or ASIC, one can use multiple wordlength paradigm

Optimal word-lengthLet Z be a realization and β = (βZ , βT , βX , βY , βADD) theword-lengthWe consider J a computational cost (area, power consumption,WCET, etc.). The optimal wordlength problem is:

βopt = arg minβ∈B

J (β) with SQNR > SQNRmin


word-lengthoptimization


Global methodology

structure

transfer function

set of equivalentrealizations

realization

a priori measure

wordlength

optimalimplementation

fixed-point implementation

optimal realization

SIFStructurecatalog

language

code

code generation

a) b)

c)

d)

e)

f)

fixed-point realization

implementationscheme

a) Choose a structure(state-space, ρDFIIt, ...)



Global methodology

structure

transfer function


realization

a priori measure

wordlength



optimal realization

SIFStructurecatalog

language

code

code generation

a) b)

c)

d)

e)

f)



b) Formalization SIF andset of equivalentrealizations



Global methodology

structure

transfer function


realization

a priori measure

wordlength



optimal realization

SIFStructurecatalog

language

code

code generation

a) b)

c)

d)

e)

f)



c) Optimization of an apriori criterion



Global methodology

structure

transfer function


realization

a priori measure

wordlength



optimal realization

SIFStructurecatalog

language

code

code generation

a) b)

c)

d)

e)

f)



d) Given someword-lengths and ansum-of-products evaluationscheme, the fixed-pointrepresentation are defined



Global methodology

structure

transfer function


realization

a priori measure

wordlength



optimal realization

SIFStructurecatalog

language

code

code generation

a) b)

c)

d)

e)

f)



e) Optimization of theimplementation:optimization of acomputational cost undera priori criterion constraint



Global methodology

structure

transfer function


realization

a priori measure

wordlength



optimal realization

SIFStructurecatalog

language

code

code generation

a) b)

c)

d)

e)

f)



f) Fixed-point codegeneration (C, VHDL, ...)



Example – 1

An other example to be implemented on 8-bit device (with 16-bitaccumulator)

H(z) =0.02088z2 + 0.04176z + 0.02088

z2 − 1.651z + 0.7342

Some possible realizations:R1 : Direct Form IR2 : Direct Form IIR3 : Balanced state-spaceR4 : σ2

∆H -optimal state-spaceR5 : optimal state-space with δ-optimalR6 : optimal ρ-Direct Form II



Example – 1

An other example to be implemented on 8-bit device (with 16-bitaccumulator)

H(z) =0.02088z2 + 0.04176z + 0.02088

z2 − 1.651z + 0.7342

Some possible realizations:R1 : Direct Form IR2 : Direct Form IIR3 : Balanced state-spaceR4 : σ2

∆H -optimal state-spaceR5 : optimal state-space with δ-optimalR6 : optimal ρ-Direct Form II



Example – 2

Roundoff Before Multiplication - 8 bits

realisation σ2∆H ‖H − H†‖2 σ2

ξ′ Nb + ×R1 114.8242 0.024423 0.0092127 4 + 5×R2 51.0143 0.02797 0.017759 4 + 6×R3 8.7612 0.012219 0.0012579 6 + 9×R4 11.3538 0.0089503 0.0012985 6 + 9×R5 4.6303 0.0040723 0.0037008 8 + 11×R6 7.3958 0.010575 0.00064769 6 + 7×



Algorithms

Input: u: 8 bits integerOutput: y : 8 bits integerData: xn: array [1..5] of 8 bits integersData: T : 8 bits integerData: Acc: 16 bits integerbegin

// Intermediate variablesAcc ← (xn(1) ∗ −47);Acc ← Acc + (xn(2) ∗ 106);Acc ← Acc + xn(3);Acc ← Acc + (xn(4) ∗ 3);Acc ← Acc + u;T1 ← Acc >> 6;// Statesxn(1)← xn(2);xn(2)← T1;xn(3)← xn(4);xn(4)← u;// Outputsy ← T1;

end

Algorithm 1: R1 : Direct Form I

Input: u: 8 bits integerOutput: y : 8 bits integerData: xn: array [1..5] of 8 bits integersData: T : array [1..5] of 8 bits integersData: Acc: 16 bits integerbegin

// Intermediate variablesAcc ← (xn(1) ∗ −31);Acc ← Acc + (xn(2) ∗ 114);Acc ← Acc + (u ∗ −8);T1 ← Acc >> 7;Acc ← (xn(1) ∗ −64);Acc ← Acc + (xn(2) ∗ −116);Acc ← Acc + (u ∗ −114);T2 ← Acc >> 8;// StatesAcc ← T1 << 5;Acc ← Acc + xn(1) << 6;xn(1)← Acc >> 6;Acc ← T2 << 5;Acc ← Acc + xn(2) << 6;xn(2)← Acc >> 6;// OutputsAcc ← (xn(1) ∗ −92);Acc ← Acc + (xn(2) ∗ −31);Acc ← Acc + (u ∗ 3);y ← Acc >> 7;

end

Algorithm 2: R5 - δ-state-space


Realizations Parametrized filters Hardware choices Best fixed-point filter Optimization Methodology

Further workFurther work33



Outline

8 Realizations

9 Parametrized filters

10 Hardware choices

11 Best fixed-point filter

12 Optimization

13 Methodology



Research for new structures

New structuresExploit new interesting structures

• lattice and lattice wave digital filters• ρ-modal state-space• LGC et LCW-structures• sparse state-spaces• combine efficiently all the realizations• ...


Realizations


Parametrized filters – 1We are also considering filters/controllers where the coefficientsdepend on extra parameters θ

Example: 2nd order filter

H(z) =b0z2 + b1z + b2

a0z2 + a1z + a2

with• b0 = gT 2, b1 = 2gT 2, b2 = gT 2

• a0 = 4ξωcT + ω2cT

2 + 4, a1 = 2ω2cT

2 − 8,a2 = ω2

cT2 − 4ξωcT + 4

θ =

gTωcξ

or θ =

gFeFcπξ

, . . .


parametrized filters,LPV




H(z) =b0z2 + b1z + b2

a0z2 + a1z + a2

with• b0 = gT 2, b1 = 2gT 2, b2 = gT 2


2 + 4, a1 = 2ω2cT

2 − 8,a2 = ω2

cT2 − 4ξωcT + 4

θ =

gTωcξ

or θ =

gFeFcπξ

, . . .






H(z) =b0z2 + b1z + b2

a0z2 + a1z + a2

with• b0 = gT 2, b1 = 2gT 2, b2 = gT 2


2 + 4, a1 = 2ω2cT

2 − 8,a2 = ω2

cT2 − 4ξωcT + 4

θ =

gTωcξ

or θ =

gFeFcπξ

, . . .




Parametrized filters – 2

Question: What will be the impact of the quantization of theseparameters (π for example)?

So, we want to consider• the quantization of parameters θ• the computation of Z (θ†) and the associated quantization

What is the best realization for all θ ∈ Θ ?





Question: What will be the impact of the quantization of theseparameters (π for example)?So, we want to consider

• the quantization of parameters θ• the computation of Z (θ†) and the associated quantization






Question: What will be the impact of the quantization of theseparameters (π for example)?So, we want to consider

• the quantization of parameters θ• the computation of Z (θ†) and the associated quantization





Hardware choices

A lot of possibilities for the hardware implementation• Use appropriate operators (FPGA)

• use optimized tools (FloPoCo)• flat computations or dedicated operators• dedicated DSP blocks• multiple constants multiplications• etc.

• Residue Number Systems• Which basis 2k , 2k − 1, 2k + 1 ou 2k ± c• Scaling could be a problem


hardwarechoices


Hardware choices

A lot of possibilities for the hardware implementation• Use appropriate operators (FPGA)

• use optimized tools (FloPoCo)• flat computations or dedicated operators• dedicated DSP blocks• multiple constants multiplications• etc.

• Residue Number Systems• Which basis 2k , 2k − 1, 2k + 1 ou 2k ± c• Scaling could be a problem


hardwarechoices


Best fixed-point filter

Given a real transfer function H, is it possible to find the best (‖.‖2or ‖.‖∞) transfer function H† with fixed-point coefficients ?

• Transpose work done on polynomials• possible for FIR• much more complicated for IIR

• For a given structure, find the best fixed-point coefficients Z †

Very promising work, but need dedicated time


bestfixed-point

filter








bestfixed-point

filter








bestfixed-point

filter


Multi-objectives optimization

As seen before, we have a multi-objective optimization problem(performance/accuracy/computational cost)

• We are looking at different strategies:• GA for ρ-operators• approximate a priori criteria by LMI

• We would like to show where are the realizations



Methodology – 1

For software implementation, the different parts are (for themoment):

bestfixed-point

filter

a priorimeasures


adequationalgorithm

architecture



codegenerationRealizations




Methodology – 2

For hardware implementation, much more parts are required:

Realizations

bestfixed-point

filter

a priorimeasures


word-lengthoptimization



codegeneration

hardwarechoices

bestfixed-point

filter



ConclusionsConclusions44


Outline


Conclusions

ConclusionWe try to answer the following question:

For a given controller, how to produce optimal implementation ?

The main idea is to propose a global approach:• Model all the equivalent realizations• Propose some analytical robustness criteria• Model the fixed-point implementation (evaluation scheme)• Propose some precision/performance/cost criteria


Perspectives – 1A lot of work still need to be done:

• Look for new realizations (error-feedback, mix ρ and q DirectForm, etc.)

• Multi-objectives optimization• Parametrized or LPV controllers• Study different arithmetics (RNS, etc.)• Given a real controller, find the closest controller withfixed-point coefficients

• Etc.

These future work should lead to a tool suite allowing toautomatically transform a controller into fixed-point code withcontrolled numerical behavior

→ An initial matlab toolbox have been designed: the FiniteWordlength Realization Toolbox (been redesign with Python)


Perspectives – 1A lot of work still need to be done:

• Look for new realizations (error-feedback, mix ρ and q DirectForm, etc.)

• Multi-objectives optimization• Parametrized or LPV controllers• Study different arithmetics (RNS, etc.)• Given a real controller, find the closest controller withfixed-point coefficients

• Etc.

These future work should lead to a tool suite allowing toautomatically transform a controller into fixed-point code withcontrolled numerical behavior

→ An initial matlab toolbox have been designed: the FiniteWordlength Realization Toolbox (been redesign with Python)


Perspectives – 2

But also all these questions remain legitimate for floating-pointarithmetic:

How to produce correctly rounded filters/controllers ?


Any questions ?


Fixed-point Filters SIF Exemples ρDFIIt a priori Measures Roundoff noise analysis

AppendixAppendix55



Outline

14 Fixed-point

15 Filters

16 SIF

17 Exemples

18 ρDFIIt

19 a priori Measures

20 Roundoff noise analysis



Fixed-point representation

Fixed-point• A real number x is represented by

N × 2−γN : signed integer on β bits (2’s complement)γ : fixed integer (implicit)

• The quantization step q = 2−γ is fixed, the dynamic is fixed(and often limited)

21 20 2!1... ...2β−γ−2

β

γ

2−γ

s

partie entière partie fractionnaireβ − γ

−2β−γ−1

b0b1 b−1 b−γ

Back




Fixed-point• A real number x is represented byN × 2−γN : signed integer on β bits (2’s complement) – mantissaγ : fixed integer (implicit) – exponent


Example: the 1st bits of π are

011∆00100100001111110110101010001000...

Back




Fixed-point• A real number x is represented byN × 2−γN : signed integer on β bits (2’s complement) – mantissaγ : fixed integer (implicit) – exponent


Example: the 1st bits of π are

011∆00100100001111110110101010001000...

So, with β = 8, we chooseγ = 5

And then π is approximated by 101.2−5, and coded on 8 bits by theinteger 101Back



Filters

Signals• Continous-time signal: s(t), t ∈ R• Discrete-time signal: s ′(k), k ∈ Z

s ′(k) , s(kTe), Te sampling period

FilterA filter is defined by

• its impulse response• its transfer function• its frequency response



Filters

Signals• Continous-time signal: s(t), t ∈ R• Discrete-time signal: s ′(k), k ∈ Z

s ′(k) , s(kTe), Te sampling period

Hu(k) y(k)

FilterA filter is defined by

• its impulse response• its transfer function• its frequency response



Transfer function

H(z) =Y (z)

U(z)

where X (z) and Y (z) atre the Z-transform of u(k) and y(k)(H(z) is the Z-transform of the impulse response h(k))

Z-transformIt is the discrete equivalent of the Laplace-transform:

Z [x(k)] :Dcv → C

z 7→ X (z)

with

X (z) ,∞∑

k=0

x(k)z−k



Transfer function

H(z) =Y (z)

U(z)

where X (z) and Y (z) atre the Z-transform of u(k) and y(k)(H(z) is the Z-transform of the impulse response h(k))

Z-transformIt is the discrete equivalent of the Laplace-transform:

Z [x(k)] :Dcv → C

z 7→ X (z)

with

X (z) ,∞∑

k=0

x(k)z−k



Frequency response

Frequency response: H(e jω)

• for z = e jω, ω ∈ [0, 2π]

• for z = e j2π fFe with 0 6 f 6 Fe

H(z) = 1z−0.8∣∣H(e jΩ)∣∣

arg(H(e jΩ)

) !!"

!#

"

#

!"

!#

$%&'()*+,-.+/0

!"!1

!"!!

!""

!"!

!!2"

!!3#

!4"

!5#

"

67%8,-.+,&0

/9+,-:(%&;%<

=;,>*,'?@--.;%+A8,?0

Back



Intermediate variablesThe intermediate variables have been introduced to

• make explicit all the computations done• show the linking of the computations

Implicit FormThe intermediate variables are computed by:

J.T (k + 1) = M.X (k) + N.U(k)

with J lower triangular with 1 on diagonal, so• no need to compute J−1

• an intermediate variable can be computed from anotherintermediate variable that have been previously computed (inthe same step)⇒ is able to express computations such asY (k) =

(M1.

(M2... (MiUk)

))



Intermediate variablesThe intermediate variables have been introduced to

• make explicit all the computations done• show the linking of the computations

Implicit FormThe intermediate variables are computed by:

J.T (k + 1) = M.X (k) + N.U(k)

with J lower triangular with 1 on diagonal, so• no need to compute J−1

• an intermediate variable can be computed from anotherintermediate variable that have been previously computed (inthe same step)⇒ is able to express computations such asY (k) =

(M1.

(M2... (MiUk)

))From controller/filter to code: the optimal implementation problem Thibault Hilaire ([email protected]) 57/72


Opérateur δ

A realization with δ-operator is:δ[X (k)] = AδX (k) + BδU(k)

Y (k) = CδX (k) + DδU(k)δ , q−1

∆

And it is given by (SIF): I 0 0−∆I I 00 0 I

T (k + 1)X (k + 1)Y (k)

=

0 Aδ Bδ0 I 00 Cδ Dδ

T (k)X (k)U(k)



Opérateur δ

A realization with δ-operator is:δ[X (k)] = AδX (k) + BδU(k)

Y (k) = CδX (k) + DδU(k)δ , q−1

∆

And it is given by (SIF): I 0 0−∆I I 00 0 I

T (k + 1)X (k + 1)Y (k)

=

0 Aδ Bδ0 I 00 Cδ Dδ

T (k)X (k)U(k)



Cascad decomposition

!

A1 B1

C1 D1

" !

A2 B2

C2 D2

"

!

"

A1 0 B1

B2C1 A2 B2D1

D2C1 C2 D2D1

#

$

Uk Yk

X(1)k

X(2)k

Tk+1

I 0 0(0− B2

)I 0

− D2 0 I

T (k + 1)(X (k + 1)(1)

X (k + 1)(2)

)Y (k)

=

0

(C1 0

)D1

0(

A1 00 A2

) (B1

0

)0

(0 C2

)0

T (k)(X (k)(1)

X (k)(2)

)U(k)

Back




!

A1 B1

C1 D1

" !

A2 B2

C2 D2

"

!

"

A1 0 B1

B2C1 A2 B2D1

D2C1 C2 D2D1

#

$

Uk Yk

X(1)k

X(2)k

Tk+1

I 0 0(0− B2

)I 0

− D2 0 I

T(k + 1)(X (k + 1)(1)

X (k + 1)(2)

)Y (k)

=

0

(C1 0

)D1

0(

A1 00 A2

) (B1

0

)0

(0 C2

)0

T (k)(X (k)(1)

X (k)(2)

)U(k)

Back




!

A1 B1

C1 D1

" !

A2 B2

C2 D2

"

!

"

A1 0 B1

B2C1 A2 B2D1

D2C1 C2 D2D1

#

$

Uk Yk

X(1)k

X(2)k

Tk+1

I 0 0(0−B2

)I 0

−D2 0 I

T (k + 1)(X (k + 1)(1)

X(k + 1)(2)

)Y(k)

=

0

(C1 0

)D1

0(

A1 00 A2

) (B1

0

)0

(0 C2

)0

T (k)(X (k)(1)

X (k)(2)

)U(k)

Back



ρ-DFIIt – 1

It is possible to re-parametrized the transfer function:

H(z) =b0zn + b1zn−1 + . . .+ bn−1z + bn

zn + a1zn−1 + . . .+ an−1z + an

by

H(z) =β0%0(z) + β1%1(z) + . . .+ βn−1%n−1(z) + βn%n(z)

1 + α1%1(z) + . . .+ αn−1%n−11(z) + αn%n(z)

with

%i : z 7→n∏

j=i+1

ρj(z) and ρi : z 7→ z − γi

∆i

and to use the ρi -operator in a direct form II transposed.



ρ-DFIIt – 2

++ ++ +

!n !i!n!1 !1 !0

!!1n !!1

i+1 !!1i !!1

1

−α1−αi−αn−1−αn

y(k)

u(k)

w1(k)wi(k)wn(k)



ρ-DFIIt – 2

++ ++ +

!n !i!n!1 !1 !0

!!1n !!1

i+1 !!1i !!1

1

−α1−αi−αn−1−αn

y(k)

u(k)

w1(k)wi(k)wn(k)

+ !iz!1

!i!!1

i



ρ-DFIIt – 3Example : a 6th-order Butterworth filter

We looking at the transfer function relative error ‖h−h†‖∞‖h‖∞

, withrespect to the wordlengths used for the coefficients:

• Direct Form I (12 coefs)• ρ-DFIIt (18 coefs)

number of bits0 5 10 15 20 25

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

101

DFI!DFIIt

h−

h†

∞h ∞

So with equivalent precisionarea (slices)

DFI (18 bits) 1071ρDFIIt (5 bits) 206

Preliminar results for FPGA implementation (Xilink Virtex4) withResidue Number System B = 2k , 2k − 1, 2k + 1

Back



ρ-DFIIt – 3Example : a 6th-order Butterworth filter

We looking at the transfer function relative error ‖h−h†‖∞‖h‖∞

, withrespect to the wordlengths used for the coefficients:

• Direct Form I (12 coefs)• ρ-DFIIt (18 coefs)

So with equivalent precisionarea (slices)

DFI (18 bits) 1071ρDFIIt (5 bits) 206

Preliminar results for FPGA implementation (Xilink Virtex4) withResidue Number System B = 2k , 2k − 1, 2k + 1

Back



ρ-DFIIt – 4

It can be expressed with the SIF:

Z =

−1 ∆1 β0

. . . ∆2 0. . . . . .

...−1 ∆n 0

−α1 1 γ1 β1

−α2 0. . . γ2 β2

.... . . 1

. . ....

−αn 0 γn βn

1 0 . . . 0 0 . . . . . . 0 0

Back



Statistical approach – 1

An other approach is to consider that the coefficients Z aremodified in Z † = Z + ∆Z ,where ∆Zij are random independent centered variables, uniformlydistributed in −qZij

2 6 ∆Zij <qZij2 ,

and qZij are the quantization step of Zij (depends on its fixed-pointrepresentation).

Their order-2 moment is

σ2∆Zij

, E

(∆Zij)2

=q2Zij

12δZij




An other approach is to consider that the coefficients Z aremodified in Z † = Z + ∆Z ,where ∆Zij are random independent centered variables, uniformlydistributed in −qZij

2 6 ∆Zij <qZij2 ,

and qZij are the quantization step of Zij (depends on its fixed-pointrepresentation).Their order-2 moment is

σ2∆Zij

, E

(∆Zij)2

=q2Zij

12δZij




The idea behind this is that H is changed in H† = H + ∆H, where∆H is a transfer function with random variables as coefficients.So we define a new measure

σ2∆H ,

12π

∫ 2π

0E∥∥∆H

(e jω)∥∥2

F

dω

→ σ2∆H =

∥∥∥∥∂H∂Z × ΞZ

∥∥∥∥2

2

with

(ΞZ )ij ,

2−βZij

+1√

3bZijc2 (δZ )ij if Zij 6= 0

0 if Zij = 0




The idea behind this is that H is changed in H† = H + ∆H, where∆H is a transfer function with random variables as coefficients.So we define a new measure

σ2∆H ,

12π

∫ 2π

0E∥∥∆H

(e jω)∥∥2

F

dω

→ σ2∆H =

∥∥∥∥∂H∂Z × ΞZ

∥∥∥∥2

2

with

(ΞZ )ij ,

2−βZij

+1√

3bZijc2 (δZ )ij if Zij 6= 0

0 if Zij = 0




In a same way, the quantization of the coefficients changes λk inλ†k , λk + ∆λk .So we define σ2

∆|λ| by

σ2∆|λ| ,

n∑k=1

ωkE

(∆ |λk |)2Back



Roundoff noise analysis – 1

+!x(k) x!(k)x!(k) x(k)e(k)

Q[ ]

Quantized a signal x(k) is equivalent (under certain conditions) toadd a white independent noise e(k) with known moments:

Right-shift of d bits:

truncation best roundoffµe , Ee(k) 2−γ−1(1− 2−d ) 2−γ−d−1

σ2e , Ee(k)>e(k) 2−2γ

12 (1− 2−2d ) 2−2γ12 (1− 2−2d )




+!x(k) x!(k)x!(k) x(k)e(k)

Q[ ]

Quantized a signal x(k) is equivalent (under certain conditions) toadd a white independent noise e(k) with known moments:


truncation best roundoffµe , Ee(k) 2−γ−1(1− 2−d ) 2−γ−d−1

σ2e , Ee(k)>e(k) 2−2γ

12 (1− 2−2d ) 2−2γ12 (1− 2−2d )




During the implementation, the algorithm becomes:J.T (k + 1) ← M.X (k) + N.U(k)X (k + 1) ← K .T (k + 1) + P.X (k) + Q.U(k)

Y (k) ← L.T (k + 1) + R.X (k) + S .U(k)

The roundoff leads to the add of the noise ξ(k):

ξ(k) ,

ξT (k)ξX (k)ξY (k)




During the implementation, the algorithm becomes:J.T (k + 1) ← M.X (k) + N.U(k) + ξT (k)X (k + 1) ← K .T (k + 1) + P.X (k) + Q.U(k) + ξX (k)

Y (k) ← L.T (k + 1) + R.X (k) + S .U(k) + ξY (k)

The roundoff leads to the add of the noise ξ(k):

ξ(k) ,

ξT (k)ξX (k)ξY (k)



Roundoff noise analysis – 3The implemented system is then equivalent to

+

u(k) y(k)

y(k)ξ(k)ξ(k)

H

Hξ

The Signal to Quantization Noise ratio is then defined by:

SQNR =σ2

Yσ2ξ′

H1 is easily obtained, and σ2ξ′ = ‖Hξϕξ‖22 where ϕξ is such

ψξ = ϕξϕ>ξ and ψξ the covariance matrix of ξ

ϕξ only depends on implementation choices, whereas Hξ onlydepends on the choice of the realization.




+

u(k) y(k)

y(k)ξ(k)ξ(k)

H

Hξ


SQNR =σ2

Yσ2ξ′







+

u(k) y(k)

y(k)ξ(k)ξ(k)

H

Hξ


SQNR =σ2

Yσ2ξ′







+

u(k) y(k)

y(k)ξ(k)ξ(k)

H

Hξ


SQNR =σ2

Yσ2ξ′






Link with interval error – 1

+≡x(k) x(k) x(k)Q[ ]

[x(k)]

[e]

Quantized a signal x(k) is equivalent to add an interval error [e](k):


truncation best roundoffmid q

2 0rad q

2q2

with q = 2−γ−d


NEW!



+≡x(k) x(k) x(k)Q[ ]

[x(k)]

[e]

Quantized a signal x(k) is equivalent to add an interval error [e](k):


truncation best roundoffmid q

2 0rad q

2q2

with q = 2−γ−d


NEW!



The implemented system is still equivalent to

+

u(k) y(k)H

Hξ[ξ](k)[ξ](k) [y](k)

with [ξ](k) the interval error added in the system


NEW!


Link with interval error – 3 Back

Interval through a filterLet H be a SISO filter and [u](k) be an interval input, centered inu (constant), with radius rx (constant).

H[u](k) [y](k)

Then [y ](k) is an interval signal, centered in y (constant), withradius ry such as

y = H(0)u, and ry 6 ‖H‖`1 rx

H(0) is the DC-gain of the filter H.

The `1-norm or H is defined by ‖H‖`1 ,∑∞

k=0 |h(k)|, where h(k)is its impulse response.If H is a state-space (A,B,C , d), then ‖H‖`1 =

∑∞k=0

∣∣CAkB∣∣+ d .


NEW!

From controller to code

Technology

fputhe implementation

context linear lters

equivalent realizationscontext

contextlinear lters

fpufrom controllerlter

cfrom controllerlter

equivalent realizationsoutline

various constraints