Rodriguez & Granger Differential geometry of perceptual similarity [arXiv:1708.00138v1] Brain Engineering Lab Tech Report 2017.2 The differential geometry of perceptual similarity Antonio M Rodriguez 1 and Richard Granger 1 * 1 Brain Engineering Lab, Dartmouth College, Hanover NH 03755, USA *[email protected]Rodriguez A, Granger R (2017) The differential geometry of perceptual similarity. Brain Engineering Laboratory, Technical Report 2017.2. www.dartmouth.edu/~rhg/pubs/GrangerRGPEGa1.pdf Abstract Human similarity judgments are inconsistent with Euclidean, Hamming, Mahalanobis, and the majority of measures used in the extensive literatures on similarity and dissimilarity. From intrinsic properties of brain circuitry, we derive principles of perceptual metrics, showing their conformance to Riemannian geometry. As a demonstration of their utility, the perceptual metrics are shown to outperform JPEG compression. Unlike machine-learning approaches, the outperformance uses no statistics, and no learning. Beyond the incidental application to compression, the metrics offer broad explanatory accounts of empirical perceptual findings such as Tversky’s triangle inequality violations (1, 2), contradictory human judgments of identical stimuli such as speech sounds, and a broad range of other phenomena on percepts and concepts that may initially appear unrelated. The findings constitute a set of fundamental principles underlying perceptual similarity. Introduction: The fundamentals of perceptual similarity When do images look alike? All standard Euclidean (and Hamming, and Mahalanobis, and almost all other) standard measures of similarity turn out to be at odds with human similarity judgments. We spell out why this is the case, give explanatory principles, and provide an illustrative application to the widely-used JPEG compression algorithm: JPEG has been outperformed via extensive learning by neural network and ML methods, whereas we outperform it with no statistics, and no training. We show that the JPEG findings fall out as a special case of the underlying broad principles introduced here, which are applicable to a wide range of unsupervised methods that entail similarity measures. Euclidean vectors’ components are orthogonal, and thus ! a = (10000) and ! b = (00010) are equidistant from ! c = (00001) : distances ! a ! c and ! b ! c both have Hamming distances of 2, and Euclidean distances of 2 . However, considered as physical images, the right-hand positioning of the “1” values in vectors ! b and ! c render them more visually similar to each other than either is to ! a . When such “neighbor” relations within a vector are considered, then vector axes are not orthogonal, and non-Euclidean metrics can readily yield smaller distances between ! b and ! c than between ! a and ! c . Human judgments of similarity imply a particular geometric system, and as in the above simple example, it is easy to show that human similarity judgments do not conform to Euclidean, Hamming, Mahalanobis, or the other most commonly used similarity metrics; rather, we will show that they conform to Riemannian geometry.
23
Embed
The differential geometry of perceptual similarity Antonio ...rhg/pubs/GrangerRGPEGa1.pdf · methods (such as arithmetic encoding) would suffice. We focus on a specific erroneous
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
pisnotperceptuallyindependentofnororthogonaltoneighboringpixels.Asintheexamplesintheintroductionsection,neighboringpixelsareperceivedbyaviewerasbeing“closer”toeachotherthanaremoredistalpixels,andthisaltersperceivedsimilarity.Correspondingly,the64frequencybasesintheDCTarenotperceptuallyorthogonal(thoughtheyareorthogonalinEuclideanspace):someareperceptualjudgedmoresimilartoeachotherthanothersbyhumanviewers.Moreover,thesejudgmentsaredependentontheircoefficients,andthushavedifferentsimilaritiesindifferentpartsoftheDCTbasisspace.Thecurvatureofthespacethusisnotaffine,butratherRiemannian(Figure1).WeintroduceanappropriateRiemanniantreatmentofperceptualsimilarity.WeshowthattheresultingmethodcanreadilyoutperformJPEG,butmoreimportantly,ithasexplanatorypower:JPEGemergesasaspecialcaseofthegeneralmethod,andtheunderlyinggeometricprinciplesofhumanperceptionbecomemorecloselyexplained.TheRiemanniangeometricprinciplesofperceptionThreegeometricspacesAssumeanimageof64pixels(grayscale,fortemporarypedagogicalsimplicity),arrangedasan8x8array.InJPEGandallotherstandardcompressionmechanisms,theimageistreatedasanarbitrarily,butconsistently,ordered64-dimensionalvector,suchthateachvectorentrycorrespondstotheintensityatoneofthe64pixelsinthe8x8array.Thisrendersthedataintosimplevectorformat,enablingtheapplicabilityofvectorandmatrixoperations.However,itdoessoatthecostofeliminatingtheneighborrelationsamongpixelsinthephysicalspace.(Typically,the64pixelsareordered(arbitrarily)withentries1-8fromthetoprowofthearray,9-16fromthenextrow,andsoon.)Theeliminationofneighborrelationswouldbeirrelevantifhumanperceptionofapixelweremodulatedequally,ornotatall,bythecharacteristicsofneighboringanddistantpixelsalike;thisturnsoutnottobethecase.WeforwardthealternativeinwhichtheimageisdescribedintermsofaphysicalspaceΦ with3dimensions(forxandylocations,andintensity),foreachofthe64pixels.Thiscorrespondstoatransformationofthe“feature”space F intophysicalspaceΦ .Theexplicitrepresentationofphysicalpixelpositionsenablesperceptualencodingsthatusepixelpositionasaparameter.Inadditiontofeaturespace F andphysicalspaceΦ ,weintroduceathirdspace,Ψ ,whichweterm“perceptualspace,”thatincludesrepresentationofperceptualgeometricrelationsamongtheimageelements(Figure2).Transformsintothisperceptualspace,accomplishedbydifferentialgeometry,willbeshowntodirectlycorrespondtohumanperceptualsimilarityjudgments.BrainconnectomesareRiemannianFigure2showssampleanatomicalconnectivityamongbrainregions,anditsformalproperties.Figure2ashowsaninstanceoftypicalmammalianthalamocorticalandcortico-corticalsynapticprojections(16-20).Theprojectionpatternfromonecellularassemblytoanotherisnotperfectly“pointtopoint”(i.e.,eachcellprojectingtoexactlyonetopographicallycorrespondingtargetcell)norcompletelydiffuse(withnotopography);rather,theprojectionis“radiallyextended,”suchthateachelementcontactsarangeoftargetsroughlywithinaspatialneighborhoodorradiusaroundatarget.Figure2bshowsasimplevectorencodingoftheseprojectionpatternswithcorrespondingsynapticweights
′n , ′′n , etc.Figure2cshowsexamplesoftypicalphysiologicalneuralresponsesinearlysensorycorticalareasthatcanarisefromtheseconnectivitypatterns.Figure2dcontainsthegeneralformofaJacobianmatrixdenotingtheoveralleffectofactivityintheneuronsofaninputarea x ontheneuronsintargetarea f ;eachentryintheJacobiandesignatesthechangeinanelementof f asaconsequenceofagivenchangeinanelementof x .Figure2eisanexampleinstanceofsuchaJacobian,correspondingtothesynapticconnectivitypatterninFigure2b.Intuitively,aJacobianencodestheinteractionsamongstimuluscomponents.Ifavectorcontainedpurelyindependententries(asinanimaginedperfectlypoint-to-pointtopographywithnolateralfan-inorfan-outprojections),theJacobianwouldbetheidentitymatrix:onesalongthediagonalandallotherentrieszeros.Eachvectordimensionthenhasnoeffectonotherdimensions:agiveninputunitaffectsonlyasingletargetunit,andnoothers.Inactualconnectivity,whichdoescontainsomeradiallyextendedprojections,thereareoff-diagonalnon-zerovaluesintheJacobiancorrespondingtotheslightlynon-topographicsynapticcontacts(Figure2b).Whentheinputandoutputpatternsaretreatedasvectors,anyoff-diagonalJacobianelementsreflectinfluencesofonedimensiononothers:thedimensionsarenotorthogonal,andthevectorsareRiemannian,notEuclidean(21).Allperceptualsystemscanbeseentointrinsicallyexpressa“stance”onthegeometricrelationsthatoccuramongthecomponentsofthestimuliprocessedbythesystem.Inthedegeneratecaseofnooff-diagonalelements,thesystemwouldactasthoughitassumesindependenceofcomponents(Euclideanvectors).Inallpathwayscharacteristicofmostthalamo-corticalandcortico-corticalprojections,however,theprocessinginherentlyassumesnon-Euclideanneighborrelationsamongthestimuli.ItisnotablethatanybankofneuronalelementswithreceptivefieldsconsistingeitherofGaussiansoroffirstorsecondderivativesofGaussians,willhavepreciselytheeffectofcomputingthederivativesoftheinputsinjusttheformthatarisesinaJacobian(seeequation(2.3)below)(22-26).PhysiologicalneuronresponsepatternsthusappearthoroughlysuitedtoproducingtransformsintospaceswithRiemanniancurvatures.(Notably,thisimpliesthatasynapticchange(e.g.,LTP)causesspecificre-shapingofneurons’receptivefields,modifyingthecurvatureofthespaceofthetargetcellsinagivenprojectionpathway.)ThematrixJinFigure2ddescribestheparticulartransformfromaninputspacetoanoutputspace.Thisisaninstanceofspecifyingthedifferencesbetweenaperceptualinput,versusaperceptthatisreceivedviathisprojectionpathway.AperceiverwillprocessaninputasthoughitcontainstheneighborrelationsspecifiedbytheJacobian.ThemapfromphysicaltoperceptualspaceNeighboringentriesinavector,likeadjacentnotesonapianokeyboard,areclosertoeachotherthanentriesfromother,non-neighboringdimensions.Thefeaturesthusdonotconstituteindependentdimensions(or,putdifferently,thedimensionsarenotorthogonal).Inthese(extremelycommon)cases,targetperceptualdistancesarecorrectlyrenderedbyRiemannianratherthanbyEuclideanmeasures.Itisnotablethatthisnotanexceptionbut
(Supplementalsections§2.5-§2.11givesamplevaluesusedtogeneratespecificJacobiansforimageprocessing,asintheexamplesshowninFigure4).ThisJacobianenablesidentificationofadistancemetricforthefeaturespace F withrespecttoitsembeddinginphysicalspaceΦ intermsofthemetrictensor g (seeSupplementalsection§2.5forexamplesofvaluesused):
gΦ:F ( !x) = JF →ΦT ( !x) i JF →Φ( !x) (2.4)
i.e.,themetrictensorgisusingmeasuresinspaceΦ appliedtoobjectsinspace F ,or,putdifferently,thetensormeasuresdistancesin F withrespecttomeasuresinspaceΦ .Then,mappingphysicalspaceΦ toperceptualspaceΨ (viaJacobianoperator JΦ→Ψ )defineshowfeaturesinthephysicalspaceareperceivedbyaviewer,enablingaformaldescriptionofhowchangesinthephysicalimageareregisteredasperceptualchanges.SpecificconstructionoftheJacobianmapping JΦ→Ψ Informationfromthephysicalstimulusorfromtheperceiver(orboth)enablesconstructionofaJacobiantomapfromphysicalvectorstotheperceptualspaceaperceivermayuse.SuchaJacobian, JΦ→Ψ ,canbeobtaineddirectlyfromeithersynapticconnectivitypatternsorfrompsychophysics–eitherbyaprioriassumptionsorfromempiricalmeasurements.(i) SynapticJacobian:
a) EmpiricalMeasureanatomicalconnectionsandsynapticstrengths,ifknown;theJacobianisdirectlyobtainedfromthosedataasinFigure2.Thesemeasurestypicallyareunavailable,butaswillbeseen,approximationsmaybedrawnfromasetofsimpleconnectivityassumptions.
b) EstimatedAssumeGaussianfall-offofrelatednessofneighboringpixelsinastimulus;measureconstituentfeaturesasin(iia)andestimateafactorbywhichdistancesbetweenstimulusinputfeatures(e.g.,pixeldistancesinxandydirections)influencetherelatednessofthefeatures(andtheresultingcurvatureoftheRiemannianspaceinwhichtheyareassumedtobeperceptuallyembedded).
wherethe !γ i termsaretheeigenvectorsoftheLaplacianofthepositionterm,eq.(2.11),
andwherethe ci termscorrespondtothecoefficientsoftheeigenvectorbasisoftheinitialconditionofthestate Ω(x,s) correspondingtotheinitialimageitself, Ω(x,0) .(Preciseformulationofthe ci isshowninthenextsection).The64solutionsoftheFourierdecompositionformthebasisspaceintowhichtheimagewillbeprojected.(ForJPEG,thisisthediscretecosinetransformorDCTset,asmentioned;wewillseethatthiscorrespondstoonespecialcaseofthesolution,foraspecificsetofvaluesoftheentropyconstraintequation.)ApplicationofentropyconstraintequationtoimagefeaturespaceConsiderthegraph(Figure4a)whosenodesaredimensionsoffeaturespace F andwhoseedgesarethepairwiseRiemanniandistancesbetweenthosedimensionsasdefinedbythedistancematrixofequation(2.6)insectionIIIc.Thedistancematrixcanbetreatedasthe
!A withrowindices i = 1,…,m andcolumnindices j = 1,…,n .ThegraphLaplacianis
!Lg =
!D −!A ,andthenormalizedgraphLaplacianisthen L = D 1
2 Lg D 12 .
ThetotalenergyofthesystemcanbeexpressedintermsoftheHamiltonian H ,takingtheform H = L+ P whereListheLaplacianandP(correspondingtopotentialenergy)canbeneglectedasaconstantforthepresentcase;thehamiltonianisthusequivalentforthispurposetothelaplacian:
H =
∂2Ω(x,s)
∂x 2 (2.13)
Intuitively,theHamiltonianexpressesthetradeoffsamongdifferentpossiblestatesofthesystem(Figure4);appliedtoimages,theHamiltoniancanbemeasuredforitserrors(distancefromtheoriginal)ononehand,anditsentropyorcompactnessontheother:amorecompactstate(lowerentropy)willbelessexact(highererror),andviceversa.Theaimistoidentifyanoperatorthatbeginswithapointinfeaturespace(animage)andmovesittoanotherpointsuchthatthechangesinerrorandentropycanbedirectlymeasurednotinfeaturespacebutinperceptualspace(Fig3).Thusthedesiredoperatorwillmovetheimagefromitsinitialstate(withzero“error,”sinceitistheoriginalimage,andaninitialentropyvaluecorrespondingtotheinformationintheimagestate)toanewstatewithanewtradeoffbetweenthenow-increasederrorandcorrespondingentropydecrease.TheHamiltonianenablesformulationofsuchanoperator.TheeigenvectorsoftheHamiltonian(Figure4c)constituteacandidatebasissetfortheimagevector(Figure4d),andsince HΩ = λΩ ,theeigenvaluesλ oftheHamiltoniancanprovideanoperator U (s) correspondingtoanygivendesiredentropys(seeSupplementalsection§2.3).Aswewill
updateoperator, U (s) ,tobethematrixcomposedofcolumnvectors !φ i(s) .(Each
!φ i(s)
hasonlyasinglenon-zeroentry,inthevectorlocationindexedbyi,andthus U (s) isadiagonalmatrix).Thetransformationstepsforalteringanimagetoadegradedimagewithloweredentropyandincreasederror,then,beginswiththeimagevector (
Figure4.Treatmentofimageasgraph,andderivationofHamiltonian.(a)Basisvectorsinfeaturespace F treatedasagraphwithwhosenodesarethedimensionsofthebasisandwhoseedgesarethepairwisedistancesbetweendimensions(seeEq(2.6)).Fromthatgraph,theadjacencyanddegreematrices,andthusthegraphLaplacian,canbedirectlycomputed.(b)QmatrixforJPEG(qualitylevel50%).(c)ComputedQmatrixforRGPEG.(d)HamiltonianforJPEG.(e)HamiltonianforRGPEG(seeSupplementalsection§2.7,table§5.(f)EigenvectorsofHamiltonianforJPEG.(g)EigenvectorsofHamiltonianforRGPEG.(SeeSupplementalsections§2.7-2.11).