e-VLBI progress in the South Tasso Tzioumis ATNF, CSIRO September 2006 “ “ Towards e-VLBI” Towards e-VLBI” “e-VLBI” “e-VLBI”
e-VLBI progress in the South
Tasso Tzioumis
ATNF, CSIRO
September 2006
““Towards e-VLBI” Towards e-VLBI” “e-VLBI”“e-VLBI”
Outline
• Overview - details in other presentations/posters• Current LBA systems/developments
– Tapes - phasing out– DAS - digital filters– Disks - LBADR– Software Correlators
• Network developments• “Real” e-VLBI tests• Plans & the Future
LBA
x NTD
x New Norcia
Long baseline Array (LBA)• Telescopes in Australia
– ATNF, U.Tasmania, NASA - “core” Oz VLBI network
• + South Africa• + Telescopes in Asia Pacific region
– Japan, China, Hawaii, VLBA,… Asia Pacific Telescope (APT) (VSOP support)
• Open network (EVN model of operations)• http://www.atnf.csiro.au/vlbi
• (+ Tests with NZ - fringes ok - no large antenna)
LBA -S2 tape system
• Canadian S2 VCR-based recorder– 8 VCR recorders 128 Mbps max data rates– S2 maintenance issues (no external support)
• 7-station S2 LBA correlator at ATNF Sydney– XF architecture (ATCA correlator design)– Difficult to upgrade correlator
• The main production-VLBI system for > 12 years
• 2005 workshop: Phase-out S2 within ~3-4 years?• 2006 workshop: Decommission S2 in 2006-7!!
• Move to disk-systems + software correlation– Correlation contracted to Swinburne - 1 October 2006
LBA - DAS• DAS: Based on Digital Filters (> 10 years old!)
– 2 input IFs @ 64 MHz BW (2 pols or 2 frequencies)– Sampling (10 bits) + Digital filters (ideal passbands)– Bands: 64MHz 125 KHz– S2 mode: 2 x 16 MHz ( 128 Mbps to match S2)– Huygens mode: 4 x 16 MHz (256 Mbps)– Maximum: 2 x 64 MHz (512 Mbps) - tested in 2006– Multibit mode: 2 x 16 MHz x 8(10) bits 1 Gbps
• Interference excision?
• ATNF stations: 2 DAS units– 512 Mbps for Huygens (8 x 16 MHz)– 1 Gbps by 4 x 64 MHz channels - developed & tested in 2006
• Remains the main DAS for LBA system(s) – BUT no more units available!!
• Move to new VLBI dBBC units? (Haystack) - timescales?
LBADR - Disk recorders• (See C. Phillips poster for details)
• PCEVN-Metsahovi VSI interface system– VSIC (ECL-LVDS) & VSIB(computer)– Connects to LBA DAS– All LBA antennas equipped
• COTS computers – New Tyan “Thunder K8WE” motherboards (2005)
• Multiple PCI bus systems• Internal disks; IDE removable disks; X-RAID on fibre channel• (Replaced test systems on Dell 1600SC servers)
• Apple Xserve hardware RAID systems (7 TB)
Software correlators• See poster by S. Tingay & A. Deller
• FX software correlator(s)• Swinburne Supercomputer
– 300 Dell computers in cluster– Real time correlation (“normal” modes) - 30-50 nodes
• Smaller clusters at telescopes – (Parkes: 32 nodes; ATCA: 16 nodes)
• Cray XD-1 supercomputer in WA + FPGAs– Under development
• Very versatile & robust (can run on most systems)• Only correlators in Australia for disk systems!
“Real-time” fringe testing• Disk-systems
– Can run parallel to S2 VCRs (S2 C2 output port)– Short segment (~0.1s) data or use full recorded data– Transfer to Swinburne - (slow) ftp or fast links
• Correlate and post on web– Start correlating immediately (not all data needed)– Fringe in a few secs; web post within minutes
• Semi-automatic operation• Routine for all LBA observations
– Setup fringe tests for all observations– http://pentane.ssi.swin.edu.au/~cphillip/
Increased LBA reliability and robustness
Network connectivity -1• See talk by Shaun Amy
• AARNet3 Regional Network• Fibre tails to ATNF antennas in 2005-6
– $2M project, funded by CSIRO corporate– Fibre “tails” owned by CSIRO– “e-vlbi” tests in March 2006 - Official start June 2006!– Legal contract problems!!
• 2 x 1 Gbps connectivity – 1 operating now; 2nd circuit to be connected
• 10 Gbps (x multiple wavelengths) backbone• Good international connectivity
AARNet3 Regional Network
Network connectivity -2• Swinburne: 1 Gbps link(s) constructed
– Operation delayed by legal problems!! (again)– (10 Gbps link from Parkes?)– Enables software correlator connectivity
• Hobart: Fibre links funded and in construction (2006)– 1 Gbps - telescope ---> University– 10 Gbps to AARNet– (BUT Bass-link limited to ~0.5 Gbps?)
• WA Connectivity (Cray @ UWA)– CeNTIE: 3 x 1 Gbps BUT closes in 2006!– AARNet3 links planned– Impact on x-NTD?
• (Tidbinbilla - NASA: 2 Mbps commercial links– Future links? )
First Oz real “e-VLBI”• Date: 24 August 2006• Antenna: Parkes, Mopra, ATCA• Data rate: 128 Mbps (per station)• Bandwidth: 2x16 MHz (per station)• Processor: Dual AMD server (ie only 2 CPUs)• No disk buffering --> “e-VLBI”• Time: 10minutes+
• 1-bas test Parkes-ATCA at 256 Mbps/station
Plans - Near term
• Transition production VLBI to disk systems + software correlation (2006-7)– Investigate remote-disk recording– Acquire ~$0.5M of disks
• Extensive “real” e-VLBI tests– To Swinburne cluster correlator (in 2006)– To JIVE correlator under EXPReS (2006-7)
• Data disk formatting issues (LBA - Mk5a) - VSI-E??
– Tests to Haystack/Japan/Global - VSI-E??
• DAS replacement?
Special “e-VLBI” Instrument?
• Parkes-Mopra- Narrabri– 1 Gbps links installed– 1 Gbps data from 2 x DAS systems– Software correlation (Parkes or ATCA cluster)
Real-time VLBI instrument at 1 Gbps– Cf PTI instrument (Parkes -Tid @ 10 MHz BW)
• Very popular with non-VLBI specialists
– Fast detection observations/surveys– May operate in parallel with disk recording
• E.g. correlate 2nd 512 Mbps stream from ATNF antennas
Plans - Longer term• CABB - Broadband upgrade of ATCA
– 4 Gsamples x (8) bits 32 Gbps streams– 4 IFs per antenna; 8 antennas– Digital Polyphase filters -- > CABB Correlator
• ATCA operation in 2007-8– VLBI capability designed-in (2008?)
• Need for VLBI operation:– n x 10 Gbps connections to ATCA (Narrabri)– Broadbanding of Rx systems (Mopra, Parkes)
• Narrabri polyphase filters?
– Broadband and Fast links from non-ATNF antennas
Multiple 10 Gbps inputs & e-VLBI
2GHz
4k channels
4k channels
Multiple Zoom (>2 possible)Multiple Zoom (>2 possible)
2GHz
4k channels
4k channels
Com pound ZoomCompound Zoom
2GHz
4k channels
Simple ZoomSimple Zoom
2GHz
4k channels
Basic ConfigurationBasic Configuration
FFT
FFT
FIR
FIR
Filterbank
Fringe
Rotators
Correlators
DMUX
A 2GHz bandwidth polyphase digital filterbank with 4096 channels ... ... will fit into four XC2V6000 FPGAs
FPGA hardware may be
completely reprogrammed to produce many different filterbank
configurations, as different
observations may require.
RF, IF and Baseband Signal Path
Filterbank Architecture
A Wideband Upgrade for the Australia Telescope Compact Array
Luneburg Lens Phased Array
Luneburg Lens Phased Array
6 * 22m antenna AT Compact Array
Summary• S2 tape LBA system to be decommissioned 2006-7• LBADR - disk recorder developed• Software correlation - “real time” speeds
– Very versatile, robust and portable
• LBA transition: disk recording + software correlation• Digital DAS - need to augment/replace• 1 Gbps links to ATNF antennas operating
– Links to correlator and other telescopes in construction
• First “real” e-VLBI tests successful • Plans for eVLBI instrument (Pks-Mop-Nar)• Future: 10 Gbps links + CABB correlator