Top Banner

of 15

Saliency map presentation

Aug 07, 2018

Download

Documents

hassamali
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • 8/20/2019 Saliency map presentation

    1/38

    Introduction of Saliency

    Map

    Presenter: Chien-Chi Chen

    Advisor: Jian-Jiun Ding

    1

  • 8/20/2019 Saliency map presentation

    2/38

    Outline

    • Introduction of saliency map• Button-up approach

     – L. Itti’s approach – Frequency-tuned

     – Multi-scale contrast – Depth of eld – Spectral !esidual approach – "lo#al contrast #ased

    •  $op-do%n approach – &onte't-a%are

    • Information ma'imum – Measurin( )isual saliency #y site entropy rate

    *

  • 8/20/2019 Saliency map presentation

    3/38

    Outline

    • Introduction of saliency map• Button-up approach

     – L. Itti’s approach – Frequency-tuned

     – Multi-scale contrast – Depth of eld – Spectral !esidual approach – "lo#al contrast #ased

    •  $op-do%n approach – &onte't-a%are

    • Information ma'imum – Measurin( )isual saliency #y site entropy rate

    +

  • 8/20/2019 Saliency map presentation

    4/38

    Introduction of saliency map

    • Lo%-le)el,contrast – &olor

     – Orientation

     – Sie

     – Motion

     – Depth

    • /i(h-le)el – 0eople

     – &onte't

    Important

    2 3udd etal4

    Lo%-

    le)el

    7ithface

    detection

  • 8/20/2019 Saliency map presentation

    5/38

    Outline

    • Introduction of saliency map• Button-up approach

     – L. Itti’s approach – Frequency-tuned

     – Multi-scale contrast – Depth of eld – Spectral !esidual approach – "lo#al contrast #ased

    •  $op-do%n approach – &onte't-a%are

    • Information ma'imum – Measurin( )isual saliency #y site entropy rate

    8

  • 8/20/2019 Saliency map presentation

    6/38

    L. Itti’s approach

    • 9rchitecture: "aussian

    0yramids

    !4"4B4 ; "a#orpyramids

     for θ  < =5>428>4 65>41+8>?

  • 8/20/2019 Saliency map presentation

    7/38

    L. Itti’s approach

    • Center-surround Diference• 9chie)e center-surround di@erence throu(h across-scale

    di@erence

    • Operated denoted #y Θ: Interpolation to ner scale and point-to-point su#traction

    • One pyramid for each channel: I(σ), R(σ), G(σ), B(σ), Y(σ)%here σ ∈ A5..C is the scale

  • 8/20/2019 Saliency map presentation

    8/38

  • 8/20/2019 Saliency map presentation

    9/38

    L. Itti’s approach

    • Center-surround Diference•Color Feature Maps

    !ed-"reen and ;ello%-Blue

    Center-surround DiferenceOrientation Feature Maps

    •  

    +R-G

    +R-G+G-R

    +G-R   +B-Y

    +Y-B

    +Y-B

    +B-Y

    +B-Y

    Same c and s as %ith

    intensity

    ),(),(),,(   θ θ θ    sOcO scO   −=

     RG(c, s) = | (R(c) - G(c)) Θ (G( s) - R( s)) | BY (c, s) = | (B(c) - Y(c)) Θ (Y( s) - B( s)) |

  • 8/20/2019 Saliency map presentation

    10/38

    L. Itti’s approach

    • Norali!ation Operator• 0romotes maps %ith fe% stron(

    peaHs

    • Surpresses maps %ith manycompara#le peaHs

    1. ormaliation of map to ran(e A0… M C

    *. &ompute a)era(e m of all local ma'ima+. Find the (lo#al ma'imum  M 

    2. Multiply the map #y , M – m*

  • 8/20/2019 Saliency map presentation

    11/38

    L. Itti’s approach

    Inhi"ition o# ret

    J'ample ofOperation:

  • 8/20/2019 Saliency map presentation

    12/38

    Frequency-tuned

    1*

    Image Average

    Gaussian blur 

     L

     I a

    b

     µ 

     µ µ 

     µ 

    ( , )hc

    hc hc

    hc

     L

     I x y a

    b

    ω 

    ω ω 

    ω 

    ( , ) ( , )hc

    S x y I I x y µ ω = −

  • 8/20/2019 Saliency map presentation

    13/38

    Multi-scale contrast

    • Saliency al(orithm

    Ima(e

    Salienc

    y map

    Multi-scalecontrast

    &enter-surroundhisto(ra

    m

    &olorspatial-distri#uti

    on

    &onditional!andom

    Field

    1+

  • 8/20/2019 Saliency map presentation

    14/38

    Multi-scale contrast

    Multi-scale contrast

    • Local summation oflaplacian pyramid

    Center-surround histogra

    • Distance #et%eenhisto(rams of !"B color:

    2

    1 ( )( , ) || ( ) ( ) ||

     L l l c

    l x N x f x I I x I x∑ ∑

    ′= ∈′= − 22 ( )1( , )

    2 ( )

    i i s s i i

     s

     R R R R

     R R χ    ∑   −=

    +

    * 2

    ( )( ) arg max ( ( ), ( )) s

     R x R x R x R x χ =

    *

    2 * *

    { | ( )}

    ( , ) ( ( ), ( ))h xx s x x R x

     f x I R x R xω χ ∑   ′′ ′∈

    ′ ′µ

    12

  • 8/20/2019 Saliency map presentation

    15/38

    Multi-scale contrast

    • &olor spatial-distri#ution

    Ima(e,!

    "B

    "MMDistancefrom pi'el '

    to ima(ecenter

     $he)ariance of&oordinateof pi'el 'and y

    ( , ) ( | ) (1 ( )) (1 ( )) s xc

     f x I p c I V c D c∑µ × − × −

    18

  • 8/20/2019 Saliency map presentation

    16/38

    Multi-scale contrast

    • Jner(y term:

    • Saliency o#Kect:

    1 ,( | ) ( , ) ( , , )

     K 

    k k x x x x k x x

     E A I F a I S a a I λ ∑ ∑ ∑   ′′=

    = +

    ( , ), 0

    ( , ) 1 ( , ), 1

    k x

    k xk x

     f x I a

     F a I   f x I a

    =

    =  − =

    •  0air%isefeature:

    ,( , , ) | | exp( ) x x x x x xS a a I a a  β ′ ′ ′= − × −

    ,   || ||, 2 x x x x I I L !"#m′ ′= −

    2 1(2 || || ) x x I I β   −

    ′= < − >

    1

  • 8/20/2019 Saliency map presentation

    17/38

    Multi-scale contrast

    • &!F:

    •  $he deri)ati)e of the lo(-liHelihood%ith respect to

    1( | ) exp( ( | )) $ A I E A I 

     % = −

    * arg max log ( | ; )! !

    ! $ A I 

    λ λ λ ∑= rr r

    k λ 

    1

  • 8/20/2019 Saliency map presentation

    18/38

    Depth of eld

    • 9s the spread of sin(le lens ree'camera4 more and more lo% depth ofeld,DOF ima(es are captured.

    •  /o%e)er4 current saliency detectionmethods %orH poorly for the lo% DOFima(es.

    1

  • 8/20/2019 Saliency map presentation

    19/38

    Depth of eld

    • 9l(orithm:

    16

  • 8/20/2019 Saliency map presentation

    20/38

    Depth of eld

    • &lassication:   •  Focal 0oint: In a lo%DOF ima(e

    DO"

    !ectan(le %ith thehi(hest ed(e density4and center is initial

    focal point2( , ) ( , ) 

    S i & S i & A'   σ −

    =

    • &omposition

    9nalysis:se(mentation !e(ion

    1 2 3

    i

     A   !  

     A m# # S S '

      σ σ σ − − −

    =

    *5

  • 8/20/2019 Saliency map presentation

    21/38

    Spectral !esidual 9pproach

    • First scalin( ima(e to 2'2.

    •  $hen %e smoothed the saliency map%ith a (aussian lter (,' , < .σ 

    *1

  • 8/20/2019 Saliency map presentation

    22/38

    "lo#al contrast-#ased

    • /isto(ram #ased contrast,La#:

    2( )O N    2( ) ( )O N O !+

    Nuantiation of La#

    Jach channel toha)e 1* di@erent)alue

    312 172=

    **

  • 8/20/2019 Saliency map presentation

    23/38

    "lo#al contrast-#ased

    • !e(ion #ased contrast – Se(ment the Ima(e

     – AJcient (raph-#ased ima(ese(mentationC

    *+

  • 8/20/2019 Saliency map presentation

    24/38

    Outline

    • Introduction of saliency map• Button-up approach

     – L. Itti’s approach – Frequency-tuned

     – &enter-surround – Depth of eld – Spectral !esidual approach – "lo#al contrast #ased

    •  $op-do%n approach – &onte't-a%are• Information ma'imum

     – Measurin( )isual saliency #y site entropy rate

    *2

  • 8/20/2019 Saliency map presentation

    25/38

    &onte't-9%are

    • "oal: &on)ey the ima(e content

    *8

    Liu et al4 *55

  • 8/20/2019 Saliency map presentation

    26/38

    &onte't-9%are

    • Distance #et%een a pair of patches:

    ( , )( , )

    1 ( , )

    c"l"# i &

    i & p"si(i"! i &

    p p p p

    c p p=

    + ×

    salient

    /i(h  &∀

  • 8/20/2019 Saliency map presentation

    27/38

    &onte't-9%are

    • Distance #et%een a pair of patches:

    /i(h for P mostsimilar

    Saliency

    # )   = P most similar patches atscale # 

    1

    11 exp ( , )

    i &

     K # # # 

    i

    S p ) K   =

    = − −

  • 8/20/2019 Saliency map presentation

    28/38

    &onte't-9%are

    • Salient at: – Multiple scales  fore(round

     – Fe% scales  #acH(round

    1

    1

      M # 

    # i i

    # # 

    S S  M    =

    =   ∑Scale 1 Scale 2

  • 8/20/2019 Saliency map presentation

    29/38

    &onte't-9%are

    • Foci <

    • Include distance map

    0!iS   >

    1 ( ) f"ci

    i−

    $

    iS 

    ( )" 1 ( )i i f"ciS S i= −

  • 8/20/2019 Saliency map presentation

    30/38

    Outline

    • Introduction of saliency map• Button-up approach

     – L. Itti’s approach – Frequency-tuned

     – &enter-surround – Depth of eld – Spectral !esidual approach – "lo#al contrast #ased

    •  $op-do%n approach – &onte't-a%are• Information ma'imum

     – Measurin( )isual saliency #y site entropy rate

    +5

  • 8/20/2019 Saliency map presentation

    31/38

    Measurin( )isual saliency #ysite entropy rate

    +1

    1

  • 8/20/2019 Saliency map presentation

    32/38

    Measurin( )isual saliency #ysite entropy rate

    +*

    9 fully-connected (raph representation is adopted

    for each

    *

  • 8/20/2019 Saliency map presentation

    33/38

    Su#-#and (raphrepresentation

    ++

  • 8/20/2019 Saliency map presentation

    34/38

    Su#-#and (raphrepresentation

    +2

  • 8/20/2019 Saliency map presentation

    35/38

    Measurin( )isual saliency #ysite entropy rate

    +8

    9 random %alH is adopted on each su#-#and (raph. 9nd

    Site entropy rate,SJ! is measured the a)era(e informationfrom a node to the other

    +

  • 8/20/2019 Saliency map presentation

    36/38

     $he site entropy rate

    •  

    •  

    +

    i&i&

    i& &

     $ ω 

    ω ∑=

    , #, ,2

    i

    i i i& i& & i & & i

    * * * π ω ω ∑ ∑ >= = =

  • 8/20/2019 Saliency map presentation

    37/38

    &onclusion

    • Ima(e processin( is funny

    • Qnusual in its nei(h#orhood %illcorrespond to hi(h saliency %ei(ht

    • &ontrast is the Hey of saliency

    +

  • 8/20/2019 Saliency map presentation

    38/38

    !eference

    A1C !. 9chanta4 F. Jstrada4 0. 7ils4 and S. SRusstrunH. Salient re(iondetection and se(mentation. In I&S4 pa(es T8. Sprin(er4 *55.2154 21*4 212

    A*C !. 9chanta4 S. /emami4 F. Jstrada4 and S. SRusstrunH. Frequency-tuned salient re(ion detection. In &0!4 pa(es 186T1524 *556. 25642154 21*4 21+4 2124 218

    A+C L. Itti4 &. Poch4 and J. ie#ur. 9 model of saliency #ased )isualattention for rapid scene analysis. IJJJ $09MI4 *5,11:1*82T1*864166. 2564 2154 21*4 212

    A2C U. /ou and L. Vhan(. Saliency detection: 9 spectral residualapproach. In &0!4 pa(es 1T4 *55. 2154 21*4 21+4 212

    A8C S. "oferman4 L. VelniH-Manor4 and 9. $al. &onte't-a%are saliencydetection. In &0!4 *515. 2154 21*4 21+4 2124 218

    AC MM &hen(4 "U Vhan(4 . 3. Mitra4 U. /uan(4 S.M. /u. "lo#al &ontrast#ased Salient !e(ion Detect. In &0!4 *511 .AC $. Liu4 V. ;uan4 3. Sun4 3.7an(4 . Vhen(4 $. U.4 and S. /.;. Learnin( to

    detect a salient o#Kect. IJJJ $09MI4 ++,*:+8+T+4 *511. 215AC 7. 7an(4 ;. 7an(4 N. /uan(4 7. "ao4 Measurin( isaul Saliency #y

    Site Jntropy !ate4 In &0!4 *515.

    +