ASTRONOMY 6523 Spring 2013 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Professor: Jim Cordes Place and Time: 622 Space Sciences Building, TTh 2:55-4:10 p.m. Text: Bayesian Logical Data Analysis for the Physical Sciences, P. C. Gregory Additional References: Unpublished notes & selected articles Probability, Random Variables & Stochastic Processes, A. Papoulis Bayesian Inference in Statistical Analysis, G.E.P. Box & G. C. Tiao Probability Theory, E. Jaynes Aims of the Course: The emphasis is on statistical descriptions, analysis, detection, inference; model building and model fitting to empirical data. Techniques will be demonstrated through case studies encountered in astronomy and elsewhere and also with data challenges. Responsibilities: Attending lectures and asking questions Problem sets (analytical & computational) Short projects Term project Final oral exam Office, etc: 520 SSB, [email protected], 607 255-0608 Web Page: http://www.astro.cornell.edu/∼cordes/A6523 Written Materials: Instructor’s notes Articles from astrophysical, geophysical and engineering literature Assignments: Grading criteria include legibility, grammar, correctness, and completeness Project: Topic and Abstract: Due 12 March in written form and presented to class (5 min) In class presentation: Week 12 or 13 into the semester (∼ 15 minutes) Written report: Due during finals week; Text edited, In journal article style, Bibliography, Plots: labeled axes, Grading: legibility, grammar, correctness, completeness Computations: You can use any language or package you like (MATLAB, IDL, Python, Mathematica; C, C++, Fortran, etc.)
44
Embed
ASTRONOMY 6523 Spring 2013 Signal Modeling, Statistical ...hosting.astro.cornell.edu/~cordes/A6523/A6523_lecture_1_2013_all.… · Spring 2013 Signal Modeling, Statistical Inference
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ASTRONOMY 6523
Spring 2013Signal Modeling, Statistical Inference and Data Mining in Astrophysics
Professor: Jim Cordes
Place and Time: 622 Space Sciences Building, TTh 2:55-4:10 p.m.
Text: Bayesian Logical Data Analysis for the Physical Sciences, P. C. Gregory
• Characterizing astrophysical processes seen in time series
» Deterministic? Chaotic? Stochastic? » Markov proceses and random walks
• Population analyses and modeling » Stellar populations in the Milky Way » Statistical inference of spatial, velocity
distributions of neutron stars » Galactic model of electron-density turbulence
• Data mining in large data sets » Arecibo pulsar/transient survey (103 Terabytes) » RFI mitigation algorithms » Finding astrophysical signals of both known
and unknown types • Detection of gravitational waves using pulsars
! 5+ year data sets ! Exercises in many topics of this course
Basic Course Sections
• Linear systems & Fourier methods • Probability & Random Processes • Statistical inference
• While LSI systems are important, nonlinear systems and alternative basis functions are highly important in science and engineering
Pulsar Periodicity Search
time
Freq
uenc
y
time
DM
|FFT(f)|
FFT each DM’s time series
1/P2/
P3/
P• • •
Example Time Series and Power Spectrum for a recent PALFA discovery
(follow-up data set shown)
DM = 0 pc cm-3
DM = 217 pc cm-3
Time Series
Where is the pulsar?
Example Time Series and Power Spectrum for a recent PALFA discovery
(follow-up data set shown)
DM = 0 pc cm-3
DM = 217 pc cm-3
Time Series
Here is the pulsar
Spectral analysis as a unifying thread Signals ! Statistics
Spectral analysis: 1. Analysis of variance in a conjugate space
t " f (time and frequency domains)
u,v " " (interferometric images) • Statistical questions about the nature of the signal in
frequency space: a. Is there a signal? b. What is its frequency? c. What is the shape of the spectrum?
1. Basis functions: Sinusoids t " f Spherical harmonics ", # " l,m Wavelets time-frequency atoms Principal components the data determine the basis
The appropriate basis (often) is the one that most compactifies the signal in the conjugate domain
Spectral analysis as a unifying thread
Color coded temperature variations of the cosmic microwave background (CMB)
TCMB = 2.7 K
$T/TCMB ~ 10-5
Wilkinson Microwave Anisotropy Probe
Basis functions: spherical harmonics
TCMB = 2.7 K
$T/TCMB ~ 10-5
Wilkinson Microwave Anisotropy Probe
Detection: the CMB
J. Dunkley, et al., 2009, ApJS, 180, 306-329
Data Inference
Evidence! Confirmation
So we understand the big bang and that there is dark energy
Or maybe not:
“After scrutinizing over seven years’ worth of WMAP data, as well as data from the BOOMERanG balloon experiment in Antarctica, Penrose and Gurzadyn say they have identified a series of concentric circles within the data. These circles show regions in the microwave sky in which the range of the radiation’s temperature is markedly smaller than elsewhere. According to the researchers, the patterns correspond to gravitational waves formed by the collision of black holes in the aeon that preceded our own, and they published these claims in a paper submitted to arXiv” (Physics World).
Galaxy clustering Data from the Sloan Digital Sky Survey
SDSS galaxy distribution (Those with spectra)
Gamma-ray burst locations on the sky
Is there any clustering?
How would you test this?
“Flights within the US were grounded because of the attacks, and incoming international flights were diverted to Canada. Services resumed within a few days but it took years for the market to recover.“
From the BBC web page 04 Sept 2006
Example of a “change point”
Example of a transient event identifiable through data mining of article content:
Is there a periodicity in this time series?
• Repeat for L epochs spanning N=T/P spin periods
• N ~ 108 – 1010 cycles in one year • % P determined to
Basics of Pulsars as Clocks
• Signal average M pulses • Time-tag using template fitting
P !M&P
W
• J1909-3744: eccentricity < 0.00000013 (Jacoby et al.)
• B1937+21: P = 0.0015578064924327±0.0000000000000004 s
Phase residuals from isolated pulsars after subtracting a quadratic polynomial:
If these pulsars were simply spinning down in a smooth way, we would expect residuals that look like white noise:
Are any of these time series periodic? How can we test for periodicity?
Phase residuals from isolated pulsars after subtracting a quadratic polynomial:
If these pulsars were simply spinning down in a smooth way, we would expect residuals that look like white noise:
For these pulsars, the residuals are mostly caused by spin noise in the pulsar
Are any of these time series periodic? How can we test for periodicity?
Noise in Timing Residuals from G. Hobbs
Long period pulsars
MSPs
How Good are Pulsars as Clocks?
Clock processes are similar to random walks or Brownian motion. What are the best ways to characterize such processes?
Pulsars as Gravitational Wave Detectors
Earth
pulsar
pulses
Gravitational wave background
Gravitational wave background
The largest contribution to arrival times is on the time scale of the total data span length (~20 years for best cases)
MSP J1909-3744 P=3 ms + WD
Jacoby et al. (2005)
Weighted 'TOA = 74 ns
Shapiro delay
The best pulsar timing so far:
Correlation Function Between Pulsars
Correlation function of residuals vs angle between pulsars
Example power-law spectrum from merging supermassive black holes (Jaffe & Backer)
Estimation errors from: • dipole term from solar system