The ALMA Correlator Gianni Comoretto, J.C. Webber, A. Baudry, C.M. Broadwell, R. P. Escoffier, J.H. Greenberg, R.R. Treacy, P. Cais, B Quertier, P. Camino, A. Bos, W Gunst, Workshop on New Generation Correlators Groningen, 27th-29th June, 2006
Dec 25, 2015
The ALMA Correlator
Gianni Comoretto, J.C. Webber, A. Baudry, C.M. Broadwell,
R. P. Escoffier, J.H. Greenberg, R.R. Treacy, P. Cais, B Quertier,
P. Camino, A. Bos, W Gunst,
Workshop on New Generation Correlators
Groningen, 27th-29th June, 2006
Contents
• The ALMA interferometer
• The correlator
• Time and frequency division modes
• Characteristics and operating modes
• First test results
• Summary
The Atacama Large Millimeter Array
Interferometer
• Joined NRAO-Canada-Eso-Japan project
• 64 elements connected interferometer
• 5000 m altitude site, 1.5 mm precipitable H2O
• Total collecting area 7000 m2
• Baseline spacing => 12 Km: 25mas @ 200 GHz
• 30-950 GHz receivers in 10 frequency windows
• Compact array with smaller antennas
12m dish at VLA test site
Chainantor Site
The Atacama Large Millimeter Array
Interferometer
• 8 GHz total bandwidth – dual polarization
• 4 independently tunable IF channels
• Fringe stopping at 2nd LO
• 3 bit sampler, 2-4 GHz band
• Variable clock phase for fine delay compensation
• Digital fiber data transmission: 3 bit x 8 channels x 4 GS/s on
multicarrier fiber
• Minimum dump time:
– 1ms for autocorrelation data
– 16 ms for visibility data in continuum mode
– 500 ms for full spectroscopic modes (128M visibility
points, 1GB/s data rate)
• Low resolution spectroscopic mode needed for continuum
observations to remove spectral lines Dual sampler
The ALMA correlator
• 4 quadrants, each one processing single 2 GHz IF channel
• 4 polarization products, 64 antennas
• Correlator chip: 4K lags at 125 MHz clock, 2 bit sampling
• Correlation card: 32x32 antennas, 4 polarizations, 125 MHz clock
• 4 cards process 1/32 of the IF bandwidth: 512 cards total
• Time division and frequency division modesOne quadrant
Station rack Correlator rackpowersupply
Computer
System design evolution
• Initial design: pure lag correlator, with band selecting digital filter
– Time division architecture: each correlator plane analyzes a time segment
of the input data
– Limited frequency resolution at full BW: 64 points/IF (31 MHz)
– Full resolution (2K points) at 1/32 BW
• Retrofitted with digital filterboard, to implement a frequency division FXF
architecture
– Filterbank performs 32 point frequency division
– Each correlator plane analyzes one frequency segment
– Resolution increases by up to 32 (1 MHz at full BW)
– Filterboard mechanically, electrically and cost compatible with old digital
filter
• Full BW time division mode mantained for “continuum” observations
Correlator structure
• Antenna based design
• Antenna unit: delay compensation, filtering,
recirculating memory
• Correlator unit: 64x64x4 correlators, Long
Term Accumulator
• Minimum number of inteconnections:
– Each antenna unit sends 2 pol. data to 32
correlator units
– Each correlator unit receives data from 64
antenna units
– Point-to-point interconnections, without
data duplication
– LVDS interconnections with 250 MS/s
data rate
The ALMA correlator chip
• Full custom ASIC at 125 MHz clock
• Conventional lag type real correlator
• 2 bit full multiplication table. 4 blocks can be joined for 4 bit operation
• Individual block: 2 polarization per antenna
– 4x64 lags at full polarization (HH, HV, VH, VV)
– 2x128 lags at dual polarization (HH, VV)
– 256 lags with one polarization
• No cascading of blocks. Larger delays implemented using station based
buffer memory
• 20 bit integration and 16 bit secondary storage
Station (Memory) Card
• Antenna based, dual polarization
• Implements:
– bulk (static) geometric delay,
– correlation delay, to synthetize
large lags without inter-block
connections
– Time multiplexing modes
• Circular memory buffer: 1 ms being written while previous ms being read
• Write buffer free running, value stored at beginning of each ms
• 2 read buffers for each correlator plane.
• Different planes can correlate different data, different time interval, and/or different
delay range, changing only read pointer offset
Time division mode
• 1 ms data block divided into (up to) 32 shorter blocks
• Each block sent at 125 MS/s to one correlator plane
• At 2 GHz BW, 32 correlator planes needed: 64 spectral points total (4 pol. mode)
• At reduced BW, more planes available: each plane processed data with different
delay offset => more lags
• No interconnections between planes, all “routing” performed in memory board
Frequency division mode
• Digital filter to adapt input band to correlator band
• Each correlator plane analyzes 1/32 of the input band
• More correlator planes may analyze one slice (subchannel) with increased
resolution (zoom modes)
• Each subchannel can be arbitrarily positioned in the input band
• Different band/resolutions possible on different (or the same) portions of
the band
• Huge increase in correlator output: minumum integration time ~0.5 s
• For fast mapping in continuum mode (16 ms dump time), original time
division mode retained
Subchannel stitching
• Stitching together many spectra
with few points: Edge effects
important
• Real correlator: -> 0 at ->0 and
-> max
• Shift of spectral point barycentre:
error on gradients and line position• Shifting in frequency by ½ channel
improves phase response near
edges• Need to delete edge points :
overlap of sub-channels required
Polyphase filters not easily usable
RealImaginary
Tunable Filter Bank
• Array of digital receivers
• Each receiver selects and
downconverts a 62.5 MHz frequency
slot in the input 2 GHz band
• Delay compensation for short term
geometric delay variations
• Implemented using 16 FPGAs
(Altera Stratix2 EP2S30)
• Prone to single-event upset in
configuration RAM: internal CRC
monitor to detect configuration faults
Tunable Filter - Block diagram
• Configurable parameters: bandshape (via 2nd stage filter taps, only 2 sets used), central frequency, scale factor (gain) before final requantization
• Mode flags: bypass mode, ½ band mode, 4 bit mode, oversampling
Bandpass filter
• 2 stage FIR filter:
– decimating 1/32 band,
8 bit tap coefs
– Half band
bandshaping filter, 9
bit tap coefs
• > 55 dB stopband
• Average stopband > 60dB
• 0.3 dB passband ripple
• 93% useful bandwidth
Dynamic range and noise
• Dynamic range
– Polyphase digital LO/mixer using 6 bit
lookup tables: 0.9% loss, > 52 dB SFDR
– Filter: 55 dB stopband rejection
• Excess noise:
– 3 bit + 2 bit quantizations increase quant.
noise
– Possibility of trading BW for sensitivity
Sampler Mixer Total
Bypass 2 bit 13.6 13.6
Bypass 3 bit 4.1 4.1
Freq. division:
2 bit 4.1 0.1 13.6 18.3
4.1 0.1 ~8.0 ~12.0
2nd Quant
Modes for increased sensitivity
• Normal 2 bit
• Double Nyquist: +6%
• Requires 2 filters
• Output processed by 2 correlator planes
• In ½ bandwidth mode, single stream at 125MS/s processed by 1 corr. plane
• 4 bit: +13%
• Requires 2 filters
• Output processed in 4 correlator planes
Implementation
• Signal distribution direct point-to-point
LVCMOS
• Connections only between adjacent
chips
• Interboard connections point-to-point
PECL
• Signal integrity checkers on each data
path. Programmable signal phase
• Clock distribution using internal FPGA
PLL.
• Inter-rack connections 250 MHz LVDS
cables
• Power distribution using onboard DC-
DC converter, 48V busDigital filterboard
First tests with simulated data
• Pseudo random noise + 2 sine
lines + their harmonics due to
finite quantization
• 10 second integration
• Line strength 15 dB
• >40 dB spurious free dynamic
range
• Good alignment of subchannels
(< 0.1%) despite different power
level
Problems
• Thermal: 100C Tj, better cooling
schemes needed
• Mechanical: stiffners or higher
(2mm) board
Summary
• Connected element interferometer, using digital fiber link interconnection• Spectral line output, full polarization (4 pol channels)• Total BW 8+8 GHz, digitized as 4+4 channels 2 GHz, 3 bit• Resolution channels mode over 2 GHz IF (single quadrant):
– 31 MHz 64 fast continuum mode, full polarization – 1 MHz 2048 Frequency division mode, full band & polarization– 250 KHz 8192 Full band, single polarization– 3.8 KHz 8192 minimum band (31 MHz/IF), single polarization
• 64 antennas: 64 ACF, 2K CCF, with up to 1K frequency and polarization channels per baseline,
total of up to 128M lags • Dump time, limited by total data rate (1 GB/s): 16 ms / 500 ms. Typ. up to minutes• Dynamic range: ~50 dB• Technology: full custom ASIC for correlator, FPGA for filterbank • Correlator board size limits n. of antennas. All ACF and CCF on one board group. Scalable in
bandwidth and resolution, adding “planes”• FXF with first F stage performed using tunable filterbank