Vladlen Koltun, UC Berkeley Joint work with Sariel Har-Peled, Dezhen Song and Ken Goldberg Efficient Algorithms for Shared Camera Control.

Vladlen Koltun, UC BerkeleyVladlen Koltun, UC Berkeley

Joint work withJoint work with

Sariel Har-Peled, Dezhen Song and Ken GoldbergSariel Har-Peled, Dezhen Song and Ken Goldberg

Efficient Algorithms for Shared Efficient Algorithms for Shared Camera ControlCamera Control

Robotic WebcamerasRobotic Webcameras

• Positioned at sites of Positioned at sites of wide interestwide interest

• Pan, tilt, zoom can be Pan, tilt, zoom can be controlled on-linecontrolled on-line

• Existing Existing implementations: implementations: users queueusers queue

ShareCamShareCam

• Shared on-line Shared on-line camera controlcamera control

• Geometric Geometric algorithms for algorithms for finding the finding the optimal camera optimal camera frameframe

Related WorkRelated Work

• Shared on-line control over a single Shared on-line control over a single mechanismmechanism– Industrial robot arm [Goldberg, Chen, Industrial robot arm [Goldberg, Chen,

et al.]et al.]– Waste cleanup system [Cannon, Waste cleanup system [Cannon,

McDonald, et al.]McDonald, et al.]– Tele-Actor [Goldberg, Song, et al.]Tele-Actor [Goldberg, Song, et al.]

• Algorithms we describe are relevant Algorithms we describe are relevant in these scenarios as wellin these scenarios as well

ContributionContribution

• Song, van der Stappen and Song, van der Stappen and Goldberg (2002), O(nGoldberg (2002), O(n22))

• Our workOur work– Exact - O(nExact - O(n3/2 3/2 loglog33n)n)– εε-approximation - O(n logn)-approximation - O(n logn)

• Discretizing zoom valuesDiscretizing zoom values– Fixes the size of the camera rectangleFixes the size of the camera rectangle– Two degrees of freedom (center Two degrees of freedom (center

point)point)

Finding the Optimal FrameFinding the Optimal Frame

• Idea: Maximize user satisfactionIdea: Maximize user satisfaction• Find a way to evaluate the Find a way to evaluate the

satisfaction of a given user with a satisfaction of a given user with a given camera rectanglegiven camera rectangle

• Compute rectangle that maximizes Compute rectangle that maximizes cumulative user satisfactioncumulative user satisfaction

Individual User SatisfactionIndividual User Satisfaction

• Intersection over maximum metric:Intersection over maximum metric:

Intersection AreaIntersection Area

Maximum AreaMaximum Area

cameracamera

useruser

Individual Satisfaction Individual Satisfaction FunctionFunction

Intersection (userIntersection (userii,cam(x,y)),cam(x,y))

Maximum (userMaximum (userii,cam(x,y)),cam(x,y))

useruserii

SATSATii(x,y) =(x,y) =

(x,y)(x,y)

Individual Satisfaction Individual Satisfaction FunctionFunction

Intersection (userIntersection (userii,cam(x,y)),cam(x,y))

Maximum (userMaximum (userii,cam(x,y)),cam(x,y))SATSATii(x,y) =(x,y) =

useruserii

(x,y)(x,y)

Global Satisfaction Global Satisfaction FunctionFunction

• SAT(x,y) = ∑SAT(x,y) = ∑ii SAT SATii(x,y)(x,y)

• We are searching for (x,y) that We are searching for (x,y) that maximizes SAT(x,y)maximizes SAT(x,y)

Approximation AlgorithmApproximation Algorithm

• For a given For a given εε>0 and every >0 and every individual individual satisfaction satisfaction function, function, consider its consider its level sets at level sets at O(1/O(1/εε) heights) heights

Approximation AlgorithmApproximation Algorithm

• For a given For a given εε>0 and every >0 and every individual individual satisfaction satisfaction function, function, consider its consider its level sets at level sets at O(1/O(1/εε) heights) heights

Approximation AlgorithmApproximation Algorithm

• Construct Construct rectilinear rectilinear paths paths between the between the level setslevel sets

• Decompose Decompose into into rectanglesrectangles

Approximation AlgorithmApproximation Algorithm

• The weighted The weighted rectangles rectangles “almost” “almost” approximate approximate the the satisfaction satisfaction functionfunction

• We have a collection of weighted We have a collection of weighted rectanglesrectangles

• We locate the most heavily covered We locate the most heavily covered pointpoint

• The global satisfaction value of this The global satisfaction value of this point is at least (1-point is at least (1-εε) of the optimum) of the optimum

Approximation AlgorithmApproximation Algorithm

• To locate the heaviest pointTo locate the heaviest point– Sweep the plane with a vertical lineSweep the plane with a vertical line– Rectangles define weighted intervals on the Rectangles define weighted intervals on the

lineline– Maintain the heaviest point on the line Maintain the heaviest point on the line

using a segment treeusing a segment tree– During the sweep, intervals are inserted During the sweep, intervals are inserted

and removedand removed

Approximation AlgorithmApproximation Algorithm

AnalysisAnalysis

• The number of slices is O( log(1/The number of slices is O( log(1/εε)/)/εε ))

• The number of rectangles (per The number of rectangles (per function) is O( [log(1/function) is O( [log(1/εε)/)/εε]]2 2 ))

• The overall number of rectangles is The overall number of rectangles is N = O( n [log(1/ N = O( n [log(1/εε)/)/εε]]2 2 ))

• Every update of the segment tree Every update of the segment tree during the sweep is O(log N)during the sweep is O(log N)

• Overall running time is O(N log N)Overall running time is O(N log N)

Exact AlgorithmExact Algorithm

• Sweep the Sweep the satisfaction satisfaction functions with a functions with a vertical linevertical line

• Maintain the Maintain the global global satisfaction satisfaction function along function along the line with a the line with a KDSKDS

Exact AlgorithmExact Algorithm

• To avoid handling O(nTo avoid handling O(n22) events, ) events, partition points into npartition points into n1/21/2 batches batches

• Each batch maintains an additional Each batch maintains an additional value: the sum of value: the sum of longlong functions functions

• When a long function changes we When a long function changes we just update this valuejust update this value

• When a short function changes we When a short function changes we rebuild the kinetic data structurerebuild the kinetic data structure

Exact AlgorithmExact Algorithm

• The number of events processed The number of events processed by each kinetic data structure is by each kinetic data structure is ≈O(n≈O(n1/21/2))

• The kinetic data structures are The kinetic data structures are rebuilt O(n) timesrebuilt O(n) times

• The overall running time is O(nThe overall running time is O(n3/2 3/2

loglog33n)n)

ImplementationImplementation

• The approximation algorithm The approximation algorithm becomes faster than the algorithm of becomes faster than the algorithm of Song et al. only when Song et al. only when nn is in the is in the hundreds of thousands.hundreds of thousands.

• The bottleneck is updating the The bottleneck is updating the segment tree during the sweep. Each segment tree during the sweep. Each insertion or deletion takes O(log N), insertion or deletion takes O(log N), but there are 2N of them, and N is at but there are 2N of them, and N is at least 50nleast 50n

ImplementationImplementation

Work in progressWork in progress

• Hardware-assisted algorithmHardware-assisted algorithm• Continuous zoom rangeContinuous zoom range• More than one camera frameMore than one camera frame• Dynamic maintenance, obstacles, Dynamic maintenance, obstacles,

“weighted” users ...“weighted” users ...

Thank youThank you