Spit, Duct Tape, Baling Wire & Oral Tradition: Dealing With Radio Data

Spit, Duct Tape, Baling Wire & Oral Tradition:

Dealing With Radio Data

O. Smirnov (Rhodes University & SKA SA)

“A high quality radio map is a lot like a sausage, you might be curious about how it was made,

but trust me you really don't want to know.”– Jack Hickish, Oxford

O. Smirnov - SKA Challenges - SuperJEDI , Mauritius, Jul 2013 2

Radio Interferometer...

What lay people think I do What funding agenciesthink I do

What cosmologists & astrophysicists think I do What my engineers think I do What I actually do

(In celebration of the passing of an extremely lame but blissfully short-lived internet meme)

The Ron Ekers Seven-Step ProgramTo Producing A Radio Interferometer

Step 0. Admit that you have a problem:

You want to (need to/are forced to by peers/supervisors) to do interferometry.

“My name is Oleg Smirnov, and I am an interferometrist.”

How To Make An Interferometer 1

Start with a normal reflector telescope....

Then break it up into sections...

Replace the optical path with electronics

Move the electronics outside the dish

...and add cable delays

Why not drop thepieces onto the ground?

...all of them

And now replace them with proper radio dishes.

...and that's all! (?) Well almost, what about

the other pixels?

How Does Optical Imaging Do It?

This bit sees the EMF from all directions, added up together.

This bit sees the EMF from all parts of the

dish surface, added up together.∬ S l ,me i ulvmdl dm

Fourier Transforms

An optical imaging system implicitly performs two Fourier transforms:

1. Aperture EMF distribution = FT of the sky

2. Focal plane = FT-1 of the aperture EMF

A radio interferometer array measures (1) Then we do the second FT in software Hence, “aperture synthesis” imaging

The uv-Plane

Image plane

uv-plane(12 hours!)

In a sense, the two are entirely equivalent

One baseline samples one visibility at a time

Earth Rotation Aperture Synthesis

Every pair of antennas (baseline) is correlated, measures one complex visibility = one point on the uv-plane.

As the Earth rotates, a baseline sweeps out an arc in the uv-plane

See uv-coverage plot (previous slide)

Even a one-dimensional East-West array (WSRT = 14 antennas) is sufficient

Where's The Catch?

We don't measure the full uv-plane, thus we can never recover the image fully (missing information)

Interferometer = high & low-pass filter

Every visibility measurement is distorted (complex receiver gains, etc.), needs to be calibrated.

(Doesn't work the same way in optical interferometry at all...)

Can't really form up complex visibilities, etc.

Catch 1: Missing Information

Response to a point source: Point Spread Function (PSF)

PSF = FT(uv-coverage)

Observed “dirty image” is convolved with the PSF

Structure in the PSF = uncertainty in the flux distribution (corresponding to missing data in the uv-plane)

(12-hour WSRT PSF) 24

Deconvolution: from dirty to clean images

A whole continuum of skies fits the dirty image(pick any value for the missing uv-components)

Deconvolution picks one = interpolates the missing info from extra assumptions (e.g.: “sources are point-like”).

Real-life WSRT dirty image

Dirty image dominated by PSF sidelobes from the stronger sources

Deconvolution required to get at the faint stuff underneath.

Deconvolution Gone Bad

Extended sources always troublesome

Plus we're missing the zero-order spacing measurement (=total power)

...end up with a “negative bowl” problem

Ultimately, interpolating missing uv-components requires a better choice of basis functions

...and better deconvolution methods

Compressive sensing (CS) is promising

Catch 2: Measurement Errors

Incoming signal is subject to distortions (refraction, delay, amplitude loss)

atmospheric and electronic

An Uncalibrated Interferometer

Complex gain error: signal multiplied by a amplitude and phase delay term

Delay errors correspond to differences in arrival time, i.e. random shifts of antennas towards and away from the source

Amplitude errors = different sensitivities

...And Its Optical Equivalent

And The Result...

One point-like source, but observed with phase errors

In the uv-plane, phase encodes information about location

Phase errors tend to spread the flux around

Amplitude errors distort structure

And Dr Sidelobes ensures that the damage is distributed democratically

Stone-Age Calibration(First-Generation, or 1GC)

Calibrate gains using a known calibrator source Move antennas to target, cross your fingers,

and hope that everything stays stable enough to get an image

Dynamic range: ~100:1

V pq=g pqM pq

Gain of interferometer

(i.e. antenna pair) p-q

Modelvisibility

Observedvisibility

The Selfcal Revolution (2GC) Per-baseline gains are actually products of per-

antenna complex gains!

: observed visibility

: model visibility (FT of sky)

gp: antenna p complex gain

N(N-1)/2 visibilities >> N gains Start with simple M Solve for g's Improve M, rinse & repeat

dynamic range > 106:1

V pq=g p g qM pq

Typical Selfcal Cycle

Pre-calibrate g using external calibrators

Correct with g-1, make dirty image, deconvolve

Generate rough initial sky model

Solve for g using the current sky model

Correct with g-1, make dirty image, deconvolve

Optional: subtract model and work with residuals Update the sky model

Huge body of experience suggests that this works rather well, BUT there's no formal proof (!!!) Current practice is a collection of ad hoc methods, dark art and lore passed down the generations in what is virtually an oral tradition.

The Essense Of Selfcal

Essentially, selfcal is model fitting: Sky model (image of the sky): M(x,y,υ)

Instrument model (set of gains): {gp(υ,t)}

Fit this to the observed data With alternating updates of M and g

Fundamental Assumption

Basic assumption of selfcal:

every antenna sees the same (constant) sky, but has its own (time-variable) complex gain term.

V pq=g p g qM pq

The Past: Massive Overengineering(Built For 1GC, used with 2GC)

The Future: Four Sticks In The Ground (+Software)

...and Dishes Made Of Plastic(+Compatible Software)

Catch 3: Direction Dependence

Distortions on incoming signal depend on time, antenna and direction

Esp. with wide field/low frequency/high sensitivity Fortunately, have a formalism to describe this:

the RIME (Radio Interferometer Measurement Equation)

O. Smirnov - Problems of Radio Interferometric Data Reduction - FASTAR/Espresso Workshop - 30/10/2012 33

The Basics: Vectors & Jones Matrices

e= e x

v=J e= j11 j12j21 j22 ex

A dual-receptor feed measures two complex voltages (polarizations):

A transverse EM field can be described by a complex vector:

v= v x

We assume all propagation effectsare linear. Any linear transform of a vector can be described by a matrix:

Correlation

v p=J pe

vq=Jqe

vxx=⟨vpx vqx* ⟩

vyy=⟨vpy vqy*

vxy=⟨vpx vqy*

vyx=⟨vpy vqx*

⟩The same signal reaches antennas p and q along two different paths. We then correlate the two sets of complex voltages.

The 2×2 Visibility Matrix

An interferometer correlates the vectors vp ,vq :

vxx=⟨vpx vqx*

⟩ ,vxy=⟨vpx vqy*

⟩ ,vyx=⟨vpy vqx*

⟩ ,vyy=⟨vpy vqy*

Let us write this as a matrix product:

V pq=2⟨ vpvq

†⟩=2⟨vpx

vpyvqx

* vqy*

⟩=2v xx v xy

v yx v yy

( ⟨ ⟩ : time/freq averaging; † : conjugate-and-transpose)

V pq is also called the visibility matrix.

Coherencies & Stokes Parameters

Antennas p ,q measure vp= Jpe , vq= Jq e. Therefore:

V pq=2 ⟨ Jp e Jq e†⟩=2 ⟨ Jpee†

Jq†⟩= Jp2 ⟨ee†

⟩ Jq†

(making use of AB†=B† A† , and assuming Jp is constant over ⟨ ⟩)

The inner quantity is called the coherency or brightness,

and (by definition of the Stokes parameters) is actually:

B=2 ⟨ee†⟩≡ IQ UiV

U−iV I−Q

I≡⟨∣ex∣2⟩⟨∣ey∣

2⟩=⟨ex ex* ⟩⟨ey ey

* ⟩ , Q≡⟨∣ex∣2⟩−⟨∣ey∣

2⟩=⟨ex ex* ⟩−⟨ey ey

* ⟩ , etc.

And That's The RIME!

XX XYYX YY = jxx p jxy p

jyx p jyy p IQ UiV

U−iV I−Q jxx q

* jyx q

* jyy q

V pq= JpB Jq†

The RIME, in its simplest form:

measured

antenna qantenna p

source

O. Smirnov - Interferometry II & The Measurement Equation - October 2012 38

Accumulating Jones Matrices

If Jp , Jq are products of Jones matrices: Jp= Jpn ... Jp1 , Jq= Jqm ... Jq1

Since (AB)H=BH AH , the M.E. becomes:

V pq= Jpn ... Jp2 Jp1B Jq1H Jq2

H ... JqmH

or in the "onion form":

V pq= Jpn(...( Jp2( Jp1B Jq1H ) Jq2

H )...) JqmH

The Classical (2GC) Approach To Polarization Calibration

RIME version:

V pq=GpDp XDq† Gq

Scalar Equations For Polarization Selfcal

Off-Axis Effects

3C147 @21cm12h WSRT synthesis160 MHz bandwidth

22 Jy peak (3C147)13.5 μJy noise1,600,000:1 DRthermal noise-limited

Regular calibration does not reach the noise, leaves off-axis artefacts due to direction-dependenteffects (left inset)

Addressed via differential gains (right inset)

3C147 22Jy

30 mJy

26/07/11 O. Smirnov - Primary Beams, Pointing Errors & The Westerbork Wobble - CALIM2011, Manchester 42

Differential Gains, In a Nutshell

V pq= Gpgain & bandpass

differential gain

sourcecoherency

Eqs†dEq

sum over sources

dEps is frequency-independent, slowly varying in time.

Solvable for a handful of "troublesome" sources,

and set to unity for the rest.

JVLA Version

Spit, Duct Tape, Baling Wire & Oral Tradition: Dealing With Radio Data

oleg smirnov

radio interferometer

missing data

focal plane

radio data

point spread function

psf sidelobes

psf structure

Spiritual

bola baling teknik

PPD BALING-20180501112329 · SMK Agama Baling SMK Tunku...

02 bola baling

8243606 bola-baling

Permainan Baling Selipar

· bagian bawahnya. - Lepas Pengikat Baling-baling dengan....

Periodisasi Bola Baling

PERANCANGAN BALING-BALING PENGAMAN PADA KNALPOT KENDARAAN...

Mengenal Mekanisme Baling

Kemahiran Bola Baling

BALING, Oktober. Majlis Daerah Baling telah mengadakan...

Spit Master Plan: Moondarewa Spit Improvements...

Assignment Bola Baling

Baling Tin

Presona Baling Technology

Gerko Bola Baling