The Square Kilometre Array: Overview and Engineering Update

The Square Kilometre Array: Overview and Engineering Update

ALMA SCO HQ, Dec 11 2014Juande Santander-Vela

• Who am I?• SKA Science• Site, Telescopes, Technologies• SKA Organisation• Engineering Progress• SKA: the Exaflop HI Tomograph• ALMA vs. SKA: Telescope Control• What’s Next?

Outline

ALMA SCO HQ, Dec 11 2014

Who am I?


• Systems Engineer for the SDP/TM elements (since March 2014)

• VIA-SKA Project Manager (Jan ’12-Feb ’14)• Former VLT Archive Applied Scientist (Jun ’09-

Feb ’11), ALMA Science Archive Front-/Back-end Developer at ESO Garching (Mar-Dec 2011)

• Ph.D. in Radio Astronomical data archives & Virtual Observatory

• Software Analyst/Manager in Industry

Who am I?


• Introduction• Science• Site, Telescopes, Technologies• SKA as an Exaflop Camera• Engineering Progress• What’s Next?

Great observatories of the present…



…and the future



…and the future



…and the future


SKA Science


Fundamental Forces & Particles • Gravity

• Radio pulsar tests of General Relativity

• Gravitational Waves• Dark Energy

• Magnetism• Origin and evolution of

Cosmic Magnetism

Origins • Galaxies and the Universe

• Cosmic Dawn• First Galaxies• Galaxy Assembly &

Evolution• Stars, Planets & Life

• Protoplanetary Disks• Biomolecules• SETI

Why we do this? SKA Science Goals


XX Century: We discovered our place in the Universe XXI Century: We understand the Universe we inhabit

Some of these can

only be achieved

with SKA2

• Initial SKA Science Book (2004): 5 KSPs, 43 additional ones

• Updated SKA Science Book (2014): 13 Phase 1 Headline Projects, 130 Chapters• Projects reduced, parameterised from ~43

projects, through science prioritisation exercise with SKAO and SWGs

SKA Science Goals


• Astrobiology (“The Cradle of Life”) • Project Scientist: Tyler Bourke• WG Chair: Melvin Hoare

• Galaxy Evolution – Continuum • Project Scientist: Jeff Wagg• WG Chairs: Nick Seymour &

Isabella Prandoni• Cosmic Magnetism

• Project Scientist: Jimi Green• WG Chairs: Melanie Johnston-

Hollitt & Federica Govoni• Cosmology

• Project Scientist: Jeff Wagg• WG Chair: Roy Maartens

• Epoch of Reionisation & Cosmic Dawn • Project Scientist: Jeff Wagg• WG Chair: Leon Koopmans

• Galaxy Evolution – HI • Project Scientist: Jimi Green• WG Chairs: Lister Staveley-Smith

& Tom Osterloo• Pulsars (“Strong field tests of gravity”)

• Project Scientist: Jimi Green• WG Chairs: Ben Stappers &

Michael Kramer• Transients

• Project Scientist: Tyler Bourke• WG Chairs: Rob Fender & J.-P.

MacQuart

SKA Science Goals: WGs


SKA Science Goals: Headline Projects


Year in the life of SKA1: Mock-up 3-years scheduling to constrain operations, computation

Telescopes, Sites, Technologies


SKA Telescopes


SKA1-Low

SKA Telescopes


SKA1-Mid

SKA Telescopes


SKA1-Survey


SKA1-Mid: Karoo Site


SKA1-Low/-Survey: Murchison Radio Observatory

SKA Organisation


• Australia (DoI)• Canada (NRCZHIA)• China (MOST)• Germany (BMBF)• India (NCRA)• Italy (INAF)

• Netherlands (NWO)• New Zealand (MED) • South Africa (DST)• Sweden (Chalmers)• UK (STFC)

SKA Organisation


Currently UK Company Limited by Guarantee → Treaty Organisation

Other partners in negotiation, expected

SKA Governance: Pre-Construction


******************

SKA Governance

Members$

Board$of$Directors$

$SKA$Office$~50$staff$

$

Director;General$

11$Design$ConsorXa$~500$ScienXsts$&$Engineers$

Strategy$&$$Business$Dev$

Cmte$

Finance$Cmte$

ExecuXve$Cmte$

Science$&$Eng$Advisory$Cmte$

€23.4M$(cash)$

~$€125M$(in;kind)$

SKA International Design Consortia


Project Management and System Engineering Team based at JBO (UK) ~500 scientists & engineers in institutes & industry in 11 Member countries

Engineering Progress


64 x 13.5m offset Gregorian antennas8km maximum baseline lengthFirst receivers:

0.9 – 1.67 GHz (L-band)0.58 – 1 GHz (UHF)

770 MHz bandwidthEarly operations 2016/7

SKA1 Precursors: MeerKAT


First light with L-band receiver and digitisererKAT



Lot’s more of infrastructure already in place: dish assembly



Lot’s more of infrastructure already in place: dish assembly

SKA1 Precursors: ASKAP


36 x 12m antennas 3-axis movement 30m – 6km baselines Novel PAF receiver 700MHz – 1800MHz Wide field of view Fast survey speed

BETA • 6 Mk I PAFs and digital systems• Commissioning, learning about beamforming

Lesson: deploy early, test, ensure adequate gap to production commitment

ASKAP• First Mk II PAF installed• Results expected in next few weeks • Production of Mk II PAFs install through to Feb 2016

SKA1 Precursors: ASKAP


SKA1 Precursors: MWA


Murchison Widefield Array, Operational 17 refereed papers published, more coming

3 dish prototypes in testing






SKA: the Exaflop HI Tomograph


• Dishes, feeds, receivers (N=250 → 2500)• Low and mid aperture arrays (n=250k → 1000k)• Signal transport (~1 Pb/s → 10 Pb/s)• Signal processing (exa-MACs)• Software engineering & algorithm development• High performance computing (exa-flop

capability)• Data storage (exa-byte capacity)• (Distributed) power requirements (10 → 50MW)

Engineering Challenges


SKA High-Level Architecture


SKA-TEL-SKO-DD-001 Revision : 1

2013-03-12 Page 14 of 98

Figure 1 A schematic diagram of the SKA Observatory, showing the geographical locations of site entities (telescopes), the entities at regional centres (Host Country Headquarters and Science Data Processing), and

entities that are globally located (Global Headquarters).

Figure 1 shows the major SKA Observatory entities: SKA1-low and SKA1-survey in Australia, SKA1-mid in South Africa. In each Host Country the Host Country Headquarters will be in Perth and Cape Town respectively, as will the Science Data Processing Centres. The Host Country Headquarters will be responsible for Operations and Maintenance. The thick flow-lines show the uni-directional transport of large amounts of digitised data from the receptors to the central signal processing facilities on the sites, and from the central signal processing facilities to the Science Data Processing Centres. The thin dash-dot lines show the bi-directional transport of system monitor and control data.

The Science Data Processor is envisaged to be a supercomputing facility. Sufficient on-site processing will be done to reduce to a manageable rate the data sent to the off-site science computing facility. The science data processor is where calibration of the data takes place, images of sky brightness are formed, and further analysis of time-domain effects are carried out. For current aperture synthesis arrays, algorithms for carrying out calibration and imaging are mature. However, the SKA is likely to require significant new developments in this area to handle the much larger amount of data, and to achieve dynamic range targets without continuous human input. Developments by the precursors and pathfinders are likely to be critical enabling technologies.

Some fraction of the processed data and data products will be delivered to the Regional Science and Engineering Centres, which will be globally distributed. The precise nature and volume of this data will not be defined until the telescopes are closer to operations, and even then may only be defined in a preliminary way.

2.1 SKA1-low

This telescope in the Observatory will primarily address observations of the highly redshifted 21 cm hyperfine line of neutral hydrogen from the Epoch of Reionization and earlier. It will also be well

RegionalScience &

EngineeringCentre(s)

RegionalScience &

EngineeringCentre(s)

SKA Observatory Global Headquarters

Remote stationson spiral arms

ScienceComputing

RemoteStation

RemoteStation

RemoteStation

Host Country Headquarters

Central Signal Processing

SKA1-low

SKA1-survey

Core Arrays

Australia

Remote stationson spiral arms

ScienceComputing

RemoteStation

RemoteStation

RemoteStation

Host Country Headquarters

Central Signal Processing

Mid-FreqAperture Array

(SKA2)

SKA1-mid

Core Arrays

South Africa

SKA Elements & Interfaces


INFRA

DSH

LFAA

CSP SDP

TM

Command control flow

Data flow

SaDT

Timing

Conceptual figure: it should be split by telescope



INFRA

DSH/LFAA CSP SDP TM


Data flow

SaDT

Timing


• Software Defined Networking• Allows easy partitioning, sub-arraying,

quality of service, commensality…• Software Defined Beamforming (-Low, -

Survey)• Software Defined Computing

• Data-Flow Driven Science Data Processing

SKA is a Software-Defined Telescope


Lots of computing challenges


Science Data Processor Local M&C

Science Data Processor

Telescope Manager

Cor

rela

tor /

B

eam

form

er

Data Routing Ingest

Visibility processing

Multiple Reads

Time Series Search

Multiple Reads

Data$BufferData Routing

Time Series Processing

Image Plane Processing

Data Prodcuts

Sky Models, Calibration

Parameters ...

Meta Data

Master ControllerMaster Controller Local M&C Database

Tiered Data Delivery


Performance Modelling

•  Imaging and calibration determines system sizing

•  Data is Buffered after Ingest •  Detailed analysis building on Cornwell: •  Imaging

o  AW-projection o  Plus w-snapshots o  Plus faceting (working memory size)

•  Continuum imaging always required for calibration o  Field of view to second zero of beam

•  Maximal case includes “science discovery” mode of forming high spectral-resolution image cube over primary beam

•  Commensal fast-imaging mode for slow transients

Processing maximal (Pflop)

Ingest (GB/s)

Use Case examples (GB/s)

LFAA! 100! 7300! 245!

Survey! 135! 4600! 995!

Mid! 360! 3300! 255!

Detailed analysis is much more complicated

courtesy

SKA1 Archive Size


ALMA vs. SKA: Telescope Control


• Need to find COTS-based solution• Move away from expensive hard real-time

solutions• Switched network inside products• Orchestration instead of deterministic control• Certain amount of product independence

SKA Control Problem




INFRA

DSH

LFAA

CSP SDP

TM


Data flow

SaDT

Timing


ALMA SCO HQ, Dec 11 2014Figure 1. ALMA Software Subsystems interaction diagram (source: ALMA Software documentation)

By sharing a single distributed software framework, system complexity is hidden from the developers andinterfaces are standardized, but at the same time it implies a notable sensibility to low-level changes and theway in which the different subsystems make use of shared functionalities. ALMA is unparalleled in its kind interms of the amount of involved distributed elements, thus presents some unique restrictions that may or maynot have been considered by the initial requirements and design. With the number of antennas being the mostvariable part of the system, most scalability issues will be related to it.

The current operations environment outline (see figure 2) is an evolution of the Standard Test Environment.It includes network equipment that ensures a certain level of availability and isolation of each part of the system.Servers located both at the high site technical building and the operations support facility provide operationsservices and run hardware-independent (non real-time) software. In addition, each antenna has a real-time singleboard computer called ABM (Antenna Bus Master) controlling all of the antenna’s devices, mostly through CANinterface; the Correlator and Central Local Oscillator, located in the technical building, are controlled in a similarmanner. The centralized database, which stores configuration data, observation proposals and scientific dataproduced by the on-line system, is located at the operations support facility; specialized software componentstake care of data insertion and querying. A database replica exists at the central ALMA office in Santiago andat each of the ALMA Regional Centers.

ALMA has currently (June 2012) 39 accepted antennas, with 1 or 2 new antennas delivered every month.As most of the time a few antennas are busy with diverse maintenance activities, not all delivered antennas canactually be used at the same time. For example, currently only 33 are installed at the high site, 50% of thecomplete array. All 66 antennas are expected to be delivered by 2014. The growing number of antennas requiresthe rest of the system to scale alongside. Notable identified cases are:

Proc. of SPIE Vol. 8451 84510W-2

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/25/2014 Terms of Use: http://spiedl.org/terms

CSP

SDP

SDP.PIP.*

SDP.PIP.*

TM.OBSMGT

TM.TELMGT

SDP.INGEST


TM.TELMGT

TM.OBSMGT

DSH.LMC CSP.LMC

SDP.INGEST SDP.PIP.* SDP.ARCH

SDP.LMC

Commands

Data

Monitoring/Status

CSP.CORRDSH.RECEIVER

SKA Observation interaction: SKA1-Mid


Ethe

rnet

switc

hing

Midd

lewar

e TBD

(E

PICS

/TANG

O/OP

C UA

)

DSH.LMC

SADT.DDBH

SADT.TM

CSP.LMC

SADT.SAT


TM.TELMGTTM.OBSMGT

TM.TELMOD SDP.PIP.

What’s next?


20162015 2017 2018 2019 2020 2021 2022 20232014

High/level/design

Rebaselining/submissions/(RBS)

Preliminary/Design/Review/(elements)

Detailed/design

Prototype/systems/deployed

Critical/Design/Review

SKA1/construction/approved

Procurement

SKA1/early/science

SKA1/construction

SKA2/detailed/design

Design/consortia/start

SKA2/procurementSKA2/construction/starts

SKA2/concept/development

Board/RBS/approval 3K5/Mar/15

15K22/Sep/14

1/Nov/13

PreKconstruction/Stage/1

PreKconstruction/Stage/2

Production*Readiness*Review*(4*dish*array)

Construction*Readiness*Review*(1*dish*line9up)

New/SKA/Organisation/in/place

Key/Doc/Set/&/Prospectus

Formal/negotiations

Ratification/of/Agreements Organisation/has/minimum/no./Members/with/Agreements/ratified

Agreements/in/Key/Doc/Set/signed

Andrea/Casson,/SKAO/Project/Controller,/Sept/2014

System/PDR

Integration/testing/on/site

Advanced/Instrumentation/Prog.

KEY:/Blue/=/SKA1/science/&/engineering;/orange/=/policy;/green/=/SKA2;/milestones*in*italics*=/proposals/under/discussion

The SKA1 timeline

The SKA1 timeline


Issue%Baseline%Design

Science%Assessment%Workshops

Proposal%Feedback

Level%1%Requirements%Feedback

Science%Conference

Engineering%Meeting

Engineering%Change%Proposals

Designs%&%Costs

SKAO%Review

Technical%Advice(working%groups)

SEAC

Board

Consortium%Design%Activities

Level%0%Requirements

Brief%SEAC

2013 2015Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

2014

PDR

ScienceReview%Panel

ScienceReview%Panel

!! !!

!! !!

!!!!

!!(43)

!!

!!

!!

!!

Issue%Baseline%Design

Science%Assessment%Workshops

Proposal%Feedback

Level%1%Requirements%Feedback

Science%Conference

Engineering%Meeting

Engineering%Change%Proposals

Designs%&%Costs

SKAO%Review

Technical%Advice(working%groups)

SEAC

Board

Consortium%Design%Activities

Level%0%Requirements

Brief%SEAC

2013 2015Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

2014

PDR

ScienceReview%Panel

ScienceReview%Panel

Questions?Juande Santander-Vela

[email protected]

The Square Kilometre Array: Overview and Engineering Update

Science

ska sciencealma sco

ska science site

industryalma sco hq

macquartalma sco hq

swgsalma sco hq

computationalma sco

technologiesalma sco

ska project manager