Cs h196 -- Polynomials1 A Review of Polynomial Representations and Arithmetic in Computer Algebra Systems Richard Fateman University of California Berkeley.

cs h196 -- Polynomials 1

A Review of Polynomial Representations and Arithmetic in Computer Algebra Systems

Richard Fateman

University of California

Berkeley

cs h196 -- Polynomials 2

Polynomials force basic decisions in the design of a CAS

• Computationally, nearly every “applied math” object is either a polynomial or a ratio of polynomials

• Polynomial representation(s) can make or break a system: compactness, speed, generality– Examples: Maple’s object too big

– SMP’s only-double-float

– Wasteful representation can make a computation change from RAM to Disk speed (Poisson series)

cs h196 -- Polynomials 3

Polynomial Representation

• Flexibility vs. Efficiency– Most rigid: fixed array of floating-point

numbers– Among the most flexible: hash table of

exponents, arbitrary coefficients.• (cos(z)+sqrt(2)* x^3*y^4*z^5: • key is (3,4,5), entry is (+ (cos z) (expt 2 ½))

cs h196 -- Polynomials 4

A divergence in character of the data

• Dense polynomials (1 or more vars)– 3+4*x+5*x^2+0*x^3+9*x^4– 1+2*x+3*y +4*x*y+ 5*x^2 +0*y^2 + 7*x^2*y

+ 8*x*y^2 + 9*x^2*y^2 + …

• Sparse (1 or more vars)– X^100 +3*x^10 +43– 34*x^100*y^30+1– a+b+c+d+e

cs h196 -- Polynomials 5

Dense Representation

– Set number of variables v, say v=3 for x, y, z– Set maximum degree d of monomials in each

variable (or d = max of all degrees)– All coefficients represented, even if 0.– Size of such a polynomial is (d+1)^v times the

maximum size of a coefficient– Multidimensional arrays look good.

cs h196 -- Polynomials 6

Dense Representation almost same as bignums..

– Set number of variables to 1, call it “10”– All coefficients are in the set {0,…,9}– Carry is an additional operation.

cs h196 -- Polynomials 7

Sparse Representation

– Usually new variables easy to introduce– Many variables may not occur more than once– Only some set S of non-zero coefficients

represented– Linked lists, hash tables, explicit storage of

exponents.

cs h196 -- Polynomials 8

This is not a clean-cut distinction

• When does a sparse polynomial “fill in”?

• Consider powering:– (1+x^5+x^11) is sparse– Raise to 5th power: x^55 + 5 * x^49 + 5 * x^44 +

10 * x^43 + 20 * x^38 + 10 * x^37 + 10 * x^33 + 30 * x^32 + 5 * x^31 + 30 * x^27 + 20 * x^26 + x^25 + 10 * x^22 + 30* x^21 + 5 * x^20 + 20 * x^16 + 10 * x^15 + 5 * x^11 + 10 * x^10 + 5 * x^5 + 1

– This is looking rather dense.

cs h196 -- Polynomials 9

Similarly for multivariate

• When does a sparse polynomial “fill in”?

• Consider powering:– (a+b+c) is completely sparse– Cube it: c^3 + 3 * b * c^2 + 3 * a * c^2 + 3 * b^2

* c + 6 * a * b * c + 3 * a^2 * c + b^3 + 3 * a * b^2 + 3 * a^2 * b + a^3

– This is looking completely dense.

cs h196 -- Polynomials 10

How many terms in a power?

• Dense: – (d+1)^v raised to the power n is (n*d+1)^v.

• Sparse: – assume complete sparse t term polynomial p=

x1+x2+…+xt.– Size(p^n) is binomial(t+n-1,n).

• Proof: if t=2 we have binomial theorem;– If t>2 then rewrite p= xt + (poly with 1 fewer term)– Use induction.

cs h196 -- Polynomials 11

A digression on asymptotic analysis of algorithms

• Real data is not asymptotically large– Many operations on modest-sized inputs

• Counting the operations may not be right– Are the coefficient operations constant cost?– Is arithmetic dominated by storage allocation in small

cases?

• Benchmarking helps, as does careful counting• Most asymptotically fastest algorithms in CAS are

actually not used

cs h196 -- Polynomials 12

Adding polynomials

• Uninteresting problem, generally– Merge the two inputs– Combine terms that need to be combined

• Part of multiplication: partial sums– What if the terms are not produced in sorted

form? O(n) becomes O(n log n) or if done naively (e.g. insert each term as generated) perhaps O(n^2). UGH.

cs h196 -- Polynomials 13

Multiplying polynomials - I

• The way you learned in high school– M=Size(p)*Size(q) coefficient multiplies.– How many adds? How about this: terms

combine when they don’t appear in the answer. The numbers of adds is Size(p*q)-Size(p)*Size(q).

• Comment on the use of “*” above..

• We have formulas for the size of p, size of p^2, …

cs h196 -- Polynomials 14

Multiplying polynomials - II

• Karatsuba’s algorithm– Polynomials f and g: if each has 2 or more terms– Split each: if f and g are of degree d, consider

• f = f1*x^(d/2)+f0• g= g1*x^(d/2)+g0• Note that f1, f0, g1, g0 are polys of degree d/2.• Note there are a bunch of nagging details, but it still works.

– Compute A=f1*g1, C=f0*g0 (recursively!)– Compute D=(f1+f0)*(g1+g0) (recursively!)– Compute B=D-A-C– Return A*x^d+B*x^(d/2)+C (no mults).

cs h196 -- Polynomials 15

Multiplying polynomials - II

• Karatsuba’s algorithm– Cost: in multiplications, the cost of multiplying

polys of size 2r: cost(2r) is 3*cost(r) cost(s)=s^log[2](3) = s^1.585… ; looking good.

– Cost: in adds, about 5.6*s^1.585– Costs (adds+mults) 6.6*s^1.585– Classical adds+mults) is 2*s^2

cs h196 -- Polynomials 16

Karatsuba wins for degree > 18

5 10 15 20 25 30

250

500

750

1000

1250

1500

1750

cs h196 -- Polynomials 17

Analysis potentially irrelevant

• Consider that BOTH inputs must be of the same degree, or time is wasted

• If the inputs are sparse, adding f1+f2 etc probably makes the inputs denser

• A cost factor of 3 improvement is reached at size 256.

• Coefficient operations are not really constant time anymore.

cs h196 -- Polynomials 18

Multiplication III: evaluation

• Consider H(x)=F(x)*G(x), all polys in x– H(0)=F(0)*G(0)– H(1)=F(1)*G(1)– …– If degree(F) +degree(G) = 12, then degree(H) is

12. We can completely determine, by interpolation, a polynomial of degree 12 by 13 values, H(0), …, H(12). Thus we compute the product H=F*G

cs h196 -- Polynomials 19

What does evaluation cost?

• Horner’s rule evaluates a degree d polynomial in d adds and multiplies. Assume for simplicity that degree(F)=degree(G)=d, and H is of degree 2d.

• We need (2d+1) evals of F, (2d+1) evals of G, each of cost d: (2d)(2d+1).

• We need (2d+1) multiplies.• We need to compute the interpolating polynomial,

which takes O(d^2). • Cost seems to be (2d+2)*(2d+1) +O(d^2), so it is

up above classical high-school…

cs h196 -- Polynomials 20

Can we evaluation/interpolate faster?

• We can choose any n points to evaluate at, so choose instead of 0,1, …. we can choose w, w^2, w^3, … where w = nth root of unity in some arithmetic domain.

• We can use the fast Fourier transform to do these evaluations: not in time n^2 but n*log(n).

cs h196 -- Polynomials 21

About the FFT

• Major digression, see notes for the way it can be explained in a FINITE FIELD.– Evaluation = You multiply a vector of coefficients by a

special matrix F. The form of the matrix makes it possible to do the multiplication very fast.

– Interpolation = You multiply by the inverse of F. Also fast.

• Even so, there is a start-up overhead, translating the problem as given into the right domain, translating back; rounding up the size…

cs h196 -- Polynomials 22

How to compute powers of a polynomial: RMUL

• RMUL. (repeated multiplication)– P^n = P * P ^(n-1)– Algorithm: set ans:=1

• For I:=1 to n do ans:=p*ans

• Return ans.

cs h196 -- Polynomials 23

How to compute powers of a polynomial: RSQ

• RSQ. (repeated squaring)– P^(2*n) = (P^n) * (P^n) [compute P^n once..]– P^(2*n+1) = P* P^(2*n) [reduce to prev. case]

cs h196 -- Polynomials 24

How to compute powers of a polynomial: BINOM

• BINOM(binomial expansion)– P= monomial. Fiddle with the exponents– P= (p1+{p2+ …pd}) = (a+b)^n. Compute

sum(binomial(n,i)*a^i*b^(n-i), i=0,n).– Computing b^2 by BINOM, b^3, … by RMUL

etc.– Alternative: split P into two nearly-equal

pieces.

cs h196 -- Polynomials 25

Comparing RMUL and RSQ

• Using conventional n^2 multiplication, let h=size(p). RMUL computes p, p*p, p*p^2, …

• Cost is size(p)*size(p)+ size(p)*size(p^2)…

• For dense degree d with v variables:– (d+1)^v * sum ((i*d+1)^v,i=1,n-1) <

(d+1)^(2v)*n^(v+1)/(v+1).

cs h196 -- Polynomials 26

Comparing RMUL and RSQ

• Using conventional n^2 multiplication, let’s assume n=2^j.

• Cost is size(p)^2+ size(p^2)^2+ … size(p^(2^(j-1)))^2

• For dense degree d with v variables:sum ((2^i *d+1)^v,i=1,j) < (n*(d+1))^(2*v)/(3^v-1)

This is O((n*d)^(2*v)) not O(n^v*d^(2*v)) [rmul] so RSQ loses. If we used Karatsuba multiplication, the cost is O((n*d)^(1.585*v)) which is potentially better.

cs h196 -- Polynomials 27

There’s more stuff to analyze

• RSQ vs RMUL vs. BINOM on sparse polynomials

• Given P^n, is it faster to compute P^(2*n) by squaring or by multiplying by P, P, …P n times more?

cs h196 -- Polynomials 28

FFT in a finite field

• Start with evaluating a polynomial A(x) at a point x

cs h196 -- Polynomials 29

Rewrite the evaluation

• Horner’s rule

cs h196 -- Polynomials 30

Rewrite the evaluation

• Matrix multiplication

cs h196 -- Polynomials 31

Rewrite the evaluation

• Preprocessing (requires real arith)

cs h196 -- Polynomials 32

Generalize the task

• Evaluate at many points

cs h196 -- Polynomials 33

Primitive Roots of Unity

cs h196 -- Polynomials 34

The Fourier Transform

cs h196 -- Polynomials 35

This is what we need

• The Fourier transform

• Do it Fast and it is an FFT

cs h196 -- Polynomials 36

Usual notation

cs h196 -- Polynomials 37

Define two auxiliary polynomials

cs h196 -- Polynomials 38

So that

cs h196 -- Polynomials 39

Putting the pieces together:

•

cs h196 -- Polynomials 40

Simplifications noted:

•

cs h196 -- Polynomials 41

Re-stated…

cs h196 -- Polynomials 42

Even more succinctly

• Here is the Fourier Transform again..

cs h196 -- Polynomials 43

How much time does this take?

cs h196 -- Polynomials 44

What about the inverse?

•

cs h196 -- Polynomials 45

Why is this the inverse?

cs h196 -- Polynomials 46

An example: computing a power

cs h196 -- Polynomials 47

An example: computing a power

cs h196 -- Polynomials 48