Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation Value of Information 1 st year review. UCLA 2012 Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs Co-PI Raj Rao Nadakuditi University of Michigan
16
Embed
Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs
Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs . Co-PI Raj Rao Nadakuditi University of Michigan. Research program Info-driven learning. Mission Information and Objectives. Non-commutative Info Theory. Info theoretic surrogates. Consensus learning. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Valueof
Information
1st year review. UCLA 2012
Kickoff
ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation
Valueof
Information
1st year review. UCLA 2012
Numerical computation of Non-Comm. VoI Metrics & Spectra of
Random Graphs
Co-PI Raj Rao NadakuditiUniversity of Michigan
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
Mission Informationand
Objectives
Non-commutativeInfo Theory
Info-geometric learning
Consensuslearning
Info theoreticsurrogates
Information-driven Learning. Jordan (Lead); Ertin, Fisher,
Hero, Nadakuditi
Bounds, models
and
learning algorithms
Scalable, Actionable
VoI measures
Research programInfo-driven learning
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
• Principal component analysis • Direction-finding (e.g. sniper localization) • Pre-processing/Denoising to SVM-based classification (e.g. pattern, gait & face recognition) • Regression, Matched subspace detectors• Community/Anomaly detection in networks/graphs
• Canonical Correlation Analysis• PCA-extension for fusing multiple correlated sources
• Quantify eigen-VoI (Thrust 1) and Exploit quantified uncertainty (Thrust 2) for eigen-analysis based sensor fusion and learning
Eigen-analysis methods & apps.
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Role of Non-Comm. Info theory
• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models
• “Classical” info. measures in low-dim.-large sample regime• e.g. f-divergence, Shannon mutual info., Sanov’s thm.
vs.• Non-comm. info. measures in high-dim.-relatively-small-
sample regime• Non-commutative analogs of above
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Analytical signal-plus-noise model
• Low dimensional (= k) latent signal model
• Xn is n x m noise-only Gaussian matrix
• c = n/m = # Sensors / # Samples
• Theta ~ SNR
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Empirical subspaces are unequal
• c = n/m = # Sensors / # Samples
• Theta ~ SNR, X is Gaussian
• Insight: Subspace estimates are biased!• “Large-n-large-m” versus “Small-n-large-m”
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationA non-commutative VoI metric
(beyond Gaussians)
• Xn is n x m unitarily-invariant noise-only random matrix• Theorem [N. and Benaych-Georges, 2011]:
• μ = Spectral measure of noise singular values• D = D-transform of μ “log-Fourier” transform in NCI
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Numerically computing D-transform
• Desired:• Allow continuous and discrete valued inputs• O(n log n) where n is number of singular values• Numerically stable
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Empirical VoI quantification
• Based on an eigen-gap based segment, compute non-comm VoI subspaces
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Accomplishment - I
• Uk are Chebyshev polynomials• Series coefficients computed via DCT in O(n log n)• Closed-form G transform (and hence D transform)
series expansion!• “Numerical computation of convolutions in free probability theory” (with Sheehan
Olver)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models• Impact of finite training data
• Facilitate fast, accurate performance prediction for eigen-methods!
• Transition: MATLAB toolbox
Broader Impact
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Spectra of Networks
• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability
Open problem: Predict graph spectra given degree sequence
Broader Impact: ARL CTA & ITA, ARO MURI
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information Non. Comm. Prob. for Network Science
• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability
Open Solved problem: Predict spectra of a graph given expected degree sequence
Answer: Free multiplicative convolution of degree sequence with semi-circle
“Spectra of graphs with expected degree sequence” (with Mark Newman)