Highly Scalable Packetised correlators Jason Manley CASPER workshop 2009
Dec 26, 2015
Highly ScalablePacketised correlators
Jason ManleyCASPER workshop 2009
Contents
• Where we’ve come from
• CASPER
• Correlator design• Dataflow• Engine operations• Scalability• Present and future deployments
In the beginning…
Computing backend
Analogue processor
Control & Monitoring
Big dish
Interferometry…
ATA
Green Bank 300ft
…more recently…
EVLA baseline board(Carlson, 2006)
CASPERCenter for Astronomy Signal Processing and Electronics Research
• Open-source, collaborative environment• Reusable, platform-independent gateware• Modular, upgradeable hardware• Low instrument NRE, shared by the community• Enables rapid development (8 instruments / 2 years)• Industry standard communication protocols• Low Cost
CASPER DBE concept
CASPER DBE concept
Clocking
• Clocks:
• Locally synchronous, globally asynchronous.
• Sampling clock synchronized at F engines.
• X engines each run off independent local clock.
• Synchronized using global 1pps signal at ADCs
• Propagated to X engines using out-of-band signaling on XAUI links.
• Headers labeling and timestamping 10GbE Ethernet packet data.
Correlator Overview
How does it scale?
16 pol, 2k ch, 200MHz
64 pol, 2k ch, 200MHz
256 pol, 16k ch, 512MHz
1024 pol, 16k ch, 1GHz
4096 pol, 2M ch, 1GHz
8192 pol, 2M ch, 1GHz
1
10
100
1000
10000
100000
1000000
F EnginesX Engines
1998 2000 2002 2004 2006 2008 20100
100000
200000
300000
400000
500000
600000
700000
800000
43000100000
200000
330000
760000
Xilinx Virtex Family
• Processing power doubling every two years• V4 = ½ power requirements of V2Pro*
* Manufacturers claim - Xilinx Inc.
Existing systems: PAPER - Green Bank
• 8 dual-pol antennas, 150MHz.• 1st Generation: one BEE2 & four IBOBs.• First deployment of packetized system.• Continuous upgrades.
Existing systems: Northern Cross, Medicina, Italy
• Designed and deployed in under 2 weeks.• 32 single-pol antennas, 32 MHz.• 1st Generation:
• one BEE2• eight IBOBs.
New deployments: KAT-7
• 7 dual-pol antennas, 400MHz.• Pure ROACH.• New control and monitoring interface (KATCP), coarse
delay, high speed output (less than 100ms).• Expected early 2010.
New deployments: GMRT
• 32 dual-pol inputs, 400MHz bandwidth.• Collaboration – shares design with KAT-7.
New deployments: PSA-32/64/128/256
• PAPER in South Africa, staged deployment• Up to 256 antennas, 100MHz • Generation 1.5 (IBOB+ROACH) initially,• Generation 2.5 (ROACH + ROACH-II) eventually.• First field deployment of ROACH system.• Correlator deployment starting in January!
Jason [email protected]