Data reduction for weighted and outlier- resistant clustering Leonard J. Schulman Caltech joint with Dan Feldman MIT
33

# Data reduction for weighted and outlier-resistant clustering Leonard J. Schulman Caltech joint with Dan Feldman MIT.

Jan 05, 2016

## Documents

Cynthia Chase
Welcome message from author
Transcript Data reduction for weighted and outlier-resistant

clustering

Leonard J. SchulmanCaltech

joint with

Dan FeldmanMIT Talk outline

• Clustering-type problems:– k-median– weighted k-median– k-median with m outliers (small m)– k-median with penalty (clustering with many outliers)– k-line median

• Unifying framework: tame loss functions

• Core-sets, a.k.a. -approximations

• Common existence proof and algorithm   Voronoi regions have spherical boundaries  k-Median with penalty k-Median with penalty: good for outliers

2-median clustering of a data set:

Same data set plus an outlier:

Now cluster with h-robust loss function:  Related work and our results Why are all these problems in the same paper?

In each case the objective function is a suitably tame “loss function”.

The loss in representing a point p by a center c is:

k-median: D(p) = dist(p,c)

Weighted k-median: D(p) = w · dist(p,c)

Robust k-median: D(p) = min{h, dist(p,c)}

What qualifies as a “tame” loss function? Log-Log Lipschitz (LgLgLp) condition on the loss function Many examples of LgLgLp loss functions:

Robust M-estimators in Statistics

figure: Z. Zhang Classic Data Reduction Same notion for LgLgLp loss functions k-clustering core-set for loss D Weighted-k-clustering core-set for loss D

Handling arbitrary-weight centers is the “hard part” Our main technical result

1. For every LgLgLp loss fcn D on a metric space, for every set P of n points, there is a weighted-(D,k)-core-set S of size

|S| = O(log2 n)

(In more detail: |S|=(dkO(k)/2) log2 n in Rd. For finite metrics, d=log n.)

2. S can be computed in time O(n) Sensitivity [Langberg and S, SODA’11]

The sensitivity of a point p P determines how important it is to include P in a core-set:

Why this works:If s(p) is small, then p has many “surrogates” in the data,

we can take any one of them for the core-set.If s(p) is large, then there is some C for which p alone

contributes a significant fraction of the loss, so we need to include p in any core-set.

DW(p,C)

qP DW(q,C)s(p) = maxC Total sensitivity

The total sensitivity T(P) is the sum of the sensitivities of all the points:

The total sensitivity of the problem is the maximum of T(P) over all input sets P.

Total sensitivity ~ n: cannot have small core-sets.Total sensitivity constant or polylog: there may

exist small core-sets.

T(P)=sP s(p) Small total sensitivity Small coreset Small total sensitivity Small core-set The main thing we need to do in order to produce a small core-set for weighted-k-median:

For each p P compute a good upper bound on s(p) in amortized O(1) time per point.

(Upper bound should be good enough that s(p) is small) Recursive-Robust-Median(P,k)• Input:

– A set P of n points in a metric space– An integer k 1

• Output:– A subset Q P of (n/kk) points

We prove that any two points in Q can serve as each others’ surrogates w.r.t. any query. Hence each point p Q has sensitivity s(p) O(1/|Q|).

Outer loop: Call Recursive-Robust-Median(P,k), then set P:=P-Q. Repeat until P is empty.

Total sensitivity bd: T # calls to Recursive-Robust-Median kk log n.

Algorithm for computing sensitivities The algorithm to find the (n)–size set Q: Recursive-Robust-Median: illustration

c*

c* Recursive-Robust-Median: illustration

c* A detail

Actually it’s more complicated than described because we can’t afford to look for a (1+)-approximation, or even a 2-approximation, to the best k-median of any b·n points (b constant).

Instead look for a bicriteria approximation: a 2-approximation of the best k-median of any b·n/2 points. Linear time algorithm from [F,Langberg STOC’11]. High-level intuition for the correctness of Recursive-Robust-Median

Consider any p in the “output” set Q.

If for all queries C, D(p,C) is small, then p has low sensitivity.

If there is a query C for which D(p,C) is large then in that query, all points of Q are assigned to the same center c C, and are closer to each other than to c; so they are surrogates. Thankyou appendices Many examples of LgLgLp loss functions:

Robust M-estimators in Statistics М-оценка

Huber

"fair"

Cauchy

Geman-

McClure

Welsch

Tukey

Andrews

Related Documents
##### An Efficient Approach for Outlier Detection in Wireless...
Category: Engineering
##### Robust Super-Resolution by minimizing a Gaussian-weighted L2...
Category: Technology
##### 1 james schulman - georgia state
Category: Documents
##### An improvement in history based weighted voting algorithm...
Category: Technology
##### Outlier Detection - Priciples and Techniques
Category: Documents
##### Weighted Average Cost of Capital And equivalent approaches.
Category: Documents
##### Alaska 2008 School Health Profiles Report Weighted Principal...
Category: Documents
##### Outlier Selection and One Class Classification by Jeroen...
Category: Technology
##### Review. Order of Operations Using a calculator Algebra...
Category: Documents
##### Weighted Flow graphs for statistics Edwin de Jonge NTTS...
Category: Documents
##### Weighted Score And Topsis
Category: Automotive
##### Weighted average cost of capital Timothy A. Thompson...
Category: Documents