Click here to load reader

Feb 01, 2016

Markov Random Fields & Conditional Random Fields. John Winn MSR Cambridge. Road map. Markov Random Fields What they are Uses in vision/object recognition Advantages Difficulties Conditional Random Fields What they are Further difficulties. 12. 23. X 1. X 2. X 3. 234. X 4. - PowerPoint PPT Presentation

Markov Random Fields & Conditional Random FieldsJohn Winn MSR Cambridge

Road mapMarkov Random FieldsWhat they areUses in vision/object recognitionAdvantagesDifficultiesConditional Random FieldsWhat they areFurther difficulties

Markov Random Fields

Examples of use in visionGrid-shaped MRFs for pixel labelling e.g. segmentationMRFs (e.g. stars) over part positions for pictorial structures/constellation models.

AdvantagesProbabilistic model:Captures uncertaintyNo irreversible decisionsIterative reasoningPrincipled fusing of different cuesUndirected modelAllows non-causal relationships (soft constraints)Efficient algorithms: inference now practical for MRFs with millions variables can be applied to raw pixels.

Maximum Likelihood LearningSufficient statistics of dataExpected model sufficient statistics

Difficulty I: InferenceExact inference intractable except in a few cases e.g. small modelsMust resort to approximate methodsLoopy belief propagationMCMC samplingAlpha expansion (MAP solution only)

Difficulty II: LearningGradient descent vulnerable to local minimaSlow must perform expensive inference at each iteration.Can stop inference earlyContrastive divergencePiecewise training + variantsNeed fast + accurate methods

Difficulty III: Large cliquesFor images, we want to look at patches not pairs of pixels. Therefore would like to use large cliques.Cost of inference (memory and CPU) typically exponential in clique size.Example: Field of Experts, Black + RothTraining: contrastive divergence over a week on a cluster of 50+ machines Test: Gibbs sampling very slow?

Other MRF issuesLocal minima when performing inference in high-dimensional latent spacesMRF models often require making inaccurate independence assumptions about the observations.

Conditional Random FieldsX1X212X323X4234ILafferty et al., 2001

Examples of use in visionGrid-shaped CRFs for pixel labelling (e.g. segmentation), using boosted classifiers.

Difficulty IV: CRF LearningSufficient statistics of labels given the imageExpected sufficient statistics given the image

Difficulty V: Scarcity of labelsCRF is a conditional model needs labels.Labels are expensive + increasingly hard to define.Labels are also inherently lower dimensional than the data and hence support learning fewer parameters than generative models.

****Add partition function back in*********

Welcome message from author

This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Related Documents