Linear Separability in Non-Euclidean Geometry

LinearSeparability

George M.Georgiou

Outline

The problem

Separatinghyperplane

EuclideanGeometry

Non-EuclideanGeometry

Three solutions

Conclusion

1 : 29

Linear Separability in Non-EuclideanGeometry

George M. Georgiou, Ph.D.

Computer Science DepartmentCalifornia State University, San Bernardino

March 3, 2006

georgiou@csusb.edu

mailto:georgiou@csusb.edu

LinearSeparability

George M.Georgiou

Outline

The problemLinear Separability

Geometrically


EuclideanGeometry


Three solutions

Conclusion

2 : 29

The problem

Find a geodesic that separates two given sets of points onthe Poincare disk non-Euclidean geometry model.

LinearSeparability

George M.Georgiou

Outline


Geometrically


EuclideanGeometry


Three solutions

Conclusion

2 : 29

The problem


LinearSeparability

George M.Georgiou

Outline


Geometrically


EuclideanGeometry


Three solutions

Conclusion

3 : 29

Linear Separabilityin Machine Learning

Machine Learning is branch of AI, which includes areassuch as pattern recognition and artificial neural networks.

Each point X = (x1, x2, . . . , xn) ∈ Rn (vector or object)belongs to two class C0 and C1 with labels -1 and 1,respectively.

The two classes are linearly separable if there existsW = (w0, w1, w2, . . . , wn) ∈ Rn+1 such that

w0 + w1x1 + w2x2 + . . . + wnxn> 0, if X ∈ C0 (1)

w0 + w1x1 + w2x2 + . . . + wnxn< 0, if X ∈ C1 (2)

LinearSeparability

George M.Georgiou

Outline


Geometrically


EuclideanGeometry


Three solutions

Conclusion

4 : 29

Geometrically

LinearSeparability

George M.Georgiou

Outline


Geometrically


EuclideanGeometry


Three solutions

Conclusion

5 : 29

“Linearizing” the circle

(x , y) −→ (x , y , x2 + y2) (3)

Since any circle can be written as

w0 + w1x + w2y + w3(x2 + y2) = 0 (4)

LinearSeparability

George M.Georgiou

Outline


Geometrically


EuclideanGeometry


Three solutions

Conclusion

6 : 29

Separating hyperplane

w0 + w1x1 + w2x2 + . . . + wnxn= 0n = 2, 2-D: line

n = 3, 3-D: plane

n > 3, higher dimensions: hyperplane

LinearSeparability

George M.Georgiou

Outline

The problem

SeparatinghyperplanePerceptron

Linear inequalities

EuclideanGeometry


Three solutions

Conclusion

7 : 29

Finding the separating hyperplane

A number of ways

Use the perceptron algorithm

Solve a system of linear inequalities

Use Support Vector Machine (SVM) method. It maximizesseparation between the two classes.

LinearSeparability

George M.Georgiou

Outline

The problem


Linear inequalities

EuclideanGeometry


Three solutions

Conclusion

7 : 29


A number of ways




LinearSeparability

George M.Georgiou

Outline

The problem


Linear inequalities

EuclideanGeometry


Three solutions

Conclusion

7 : 29


A number of ways




LinearSeparability

George M.Georgiou

Outline

The problem


Linear inequalities

EuclideanGeometry


Three solutions

Conclusion

8 : 29

The perceptron (Rosenblatt, 1958)

w0

f(y)

w4

w3

w2

w1

x4

x3

x2

1x1

f (y) =

1, if y > 0

−1, if y < 0(5) y =

i=n∑i=0

wi xi (6)

Perceptron Algorithm1 Start with W = 0.2 Present input vector X and compute error ε = d − f (y).3 Update W using ∆W = α ε X4 Pick another input vector X and goto to step 2.

LinearSeparability

George M.Georgiou

Outline

The problem


Linear inequalities

EuclideanGeometry


Three solutions

Conclusion

9 : 29

Solving linear inequalities

Can use linear programming, e.g. simplex method,Karmakar, and variants.

Polynomial complexity

In fact, when dimension is fixed, e.g. d=2,3 as in our case,complexity is linear in the number of input vectors.

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

10 : 29

Euclidean Geometry

Axioms of Euclid (ca. 400 BC)

I. Two points determine a unique line.

II. A line is of infinite length.

III. A circle may be described with any center at any distancefrom the center.

IV. All right angles are equal.

V. If a straight line meet two other straight lines, so as tomake the two interior angles on one side of it together lessthan two right angles, the other straight lines will meet ifproduced on that side on which the angles are less thantwo right angles.

(from H.S.M. Coxeter, Non-Euclidean Geometry)

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry

Non-EuclideanGeometryPoincare Disk

Three solutions

Conclusion

11 : 29

Non-Euclidean Geometry

Euclid’s Elements considered to be the most successfultextbook of all times.

Nicolai Lobachevsky (1829) and Janos Bolyai (1832) firstto publish geometries without the Euclid’s axiom V.

Gauss feared the “howling of the Boetians.”

Types on Non-Euclidean Geometry

Lobachevskian (hyperbolic geometry) Geometry: from apoint not on a line, an infinite number of parallel lines canbe drawn.

Riemannian (elliptic) geometry: no parallel lines can bedrawn.

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

12 : 29

Poincare disk

M

N

b

b

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

12 : 29

Poincare disk

M

N

P

b

b

b

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

13 : 29

Distance function

The distance between points z1 and z2, taken as complexnumbers, is

D(z1, z2) = | log(z1, z2;µ, λ)|, (7)

where z1 and z2 are points on the unit disk, and µ and λ arethe intersections of the geodesic defined by z1 and z2 with theunit circle. The cross ratio (z1, z2;µ, λ), is defined as

(z1, z2;µ, λ) =z1 − µ

z1 − λ

/z2 − µ

z2 − λ(8)

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

14 : 29

Art by M.C. Esher (1878–1972)Limit Circle III

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

16 : 29

The problem


LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

16 : 29

The problem


LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutionsKlein disk

Hyperboloid model

Circular Paraboloid

Conclusion

17 : 29

Three solutions

1 Map points from Poincare disk to the Klein disk, findseparating line, which maps to a desired geodesic on thePoincare disk.

2 Map points from Poincare disk to the Hyperboloid model,find separating plane, which maps to a desired geodesicon the Poincare disk.

3 Map points from Poincare disk to circular paraboloid, findseparating plane, which maps to a desired geodesic onthe Poincare disk. (This is a novel map.)

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

17 : 29

Three solutions




LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

17 : 29

Three solutions




LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

18 : 29

The Klein disk

Points, like those of the Poincare disk, are the usualEuclidean points.

Lines are the familiar straight Euclidean lines, but ofcourse are bounded but the unit circle.

Angles, unlike those in the Poincare disk, are distorted.

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

18 : 29

The Klein disk




LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

18 : 29

The Klein disk




LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

19 : 29

The two geodesics

B

A

Blue: Poincare

Red: Klein

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

20 : 29

From the Poincare to the Klein disk, and back

(0,0,1)

PK

F (x , y) =

(2x

1 + x2 + y2 ,2y

1 + x2 + y2

)(9)

F−1(x , y) =

(x

1 +√

1 − x2 − y2,

y

1 +√

1 − x2 − y2

)(10)

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

21 : 29

1st Solution

1 Map all given points from the Poincare disk to the Kleindisk

2 Find a separating line3 Map the two points of the intersection of the line and the

unit circle to the Poincare disk4 Find geodesic that corresponds to the two points

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

22 : 29

The Hyperboloid model

H = (x , y ,√

x2 + y2 + 1) : x , y ∈ R (11)

Geodesics are intersections of planes through the origin.

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

23 : 29

From Poincare to the Hyperboloid, and back

Poincare Disk

Klein Disk

(x,y,z)

P

P = r + i s (12)

F (P) =

(2r

1 − |P |2,

2s1 − |P |2

,1 + |P |2

1 − |P |2

)(13)

F−1(x , y , z) =x + i y1 + z

(14)

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

24 : 29

2nd Solution

1 Map all given points from the Poincare disk to thehyperboloid

2 Find a separating plane that passes through the origin3 Map two points on the intersection of the plane and the

hyperboloid to the Poincare disk4 Find geodesic that corresponds to the two points

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

25 : 29

The circular paraboloid I

Ω = (x , y , x2 + y2 + 1) : x , y ∈ R, x2 + y2 < 1 (15)

(x,y,z)

(x,y)

Fact: Circles on the x-y plane are projections of ellipses onf (x , y) = x2 + y2 + 1. (D. Pedoe, Geometry, 1988) showsthis for the surface (x , y , x2 + y2).

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

26 : 29

The circular paraboloid II

(x,y,z)

(x,y)

It can be shown: the ellipse on f (x , y) = x2 + y2 + 1 is theintersection of a plane that passes through the origin.

It can be shown: the corresponding circles on the x-yplane are orthogonal to the unit circle.

A geodesic on the Poincare disk maps to an arc of anellipse on Ω that lies on a plane through the origin.

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

27 : 29

3rd Solution

1 Map all given points from the Poincare disk to Ω

2 Find a separating plane that passes through the origin3 Map two points on the intersection of the plane and Ω to

the Poincare disk (just discard the z coordinate)4 Find geodesic that corresponds to the two points

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry



Hyperboloid model

Circular Paraboloid

Conclusion

28 : 29

Advantages of the 3rd solution

Mapping to and from is simple.

No division necessary

Unlike 2nd method, no overflow possible in mapping

LinearSeparability

George M.Georgiou

Outline

The problem


EuclideanGeometry


Three solutions

Conclusion

29 : 29

Conclusion

Three ways given two find a geodesic that separates twoclasses of vectors on the Poincare disk.

A new way to visualize the Poincare disk is given.

Linear Separability in Non-Euclidean Geometry

Documents

Euclidean versus Non-Euclidean Geometry -...

Beyond Euclidean geometry (PDF)

Applied Mathematics Euclidean Geometry

Chapter 4 Euclidean Geometry - Fairmont State · PDF...

Euclidean Geometry and Transformations

Belyaev, O.a. - Fundamentals of Geometry Euclidean,...

MA232A: Euclidean and non-Euclidean Geometry Michaelmas ...

EUCLIDEAN GEOMETRY -...

Geometry on a Ball Spherical Geometry… a Non-Euclidean...

NON-EUCLIDEAN GEOMETRY. The Elements of Non-Euclidean

7 euclidean&non euclidean geometry

MA232A|Euclidean and Non-Euclidean Geometry School of...