Proposal Handling Platform - Zenodo

Bruno Khélif (APC/CNRS, Paris, France),Matthias Fuessling (CTAO) for the CTA Observatory

Proposal Handling Platform

1. The CTA Observatory1.1 Project1.2 Proposals Handling

2. The APC PHP2.1 Requirements & Functionalities2.2 Design & technologies

2.3 Some screenshots

3. Conclusions and discussions

Outline

The CTA Observatory

CTA : the observatory of ground-based gamma-ray astronomy

• An observatory to observe the sky at very high energies : https://www.cta-observatory.org/– Two sites for observing the full sky– From 20GeV to 300TeV

– Using the Imaging Atmospheric Cherenkov Technique

CTA : the observatory of ground-based gamma-ray astronomy

• Project Status :

• Proposal-driven open observatory– Operations carried by observatory operators– After a proprietary period, data will be made openly available

● Guest Observer (GO) programs: via Announcement of Opportunity (AO) calls

● Key Science projects (by the CTA Consortium)● Director's Discretionary Time (DDT)● Archive Access

– Use of a Time Allocation Committee (TAC) for the reviewing process → Need of a Proposal Handling system

Proposal Handling Systems : features

• Web-based application• Robustness

• User friendly

• Ensuring information safety– Users and TAC (password, proposals, reports)– Backward compatibility for the foreseen evolution of the Data Model

– 24h/24, 7d/week, automatic switch in case of failure– Should be able to support numerous connections few days

before the closing of the AO call– Different level of access rights (GO, TAC, admin, etc)

– Intuitive, easiness to find quickly documentations, tools– Keep the graphics whatever the web browser (web

responsive)– Having a User Support

→ Interface with the CTA Archive system

→ Interface with the CTA web portal→ Interface with the CTA User Support

→ Interface with the CTA informatics resources

Insertions within the CTA Top Level Use Cases

• Achievement of the Top Level Use Cases

• Define the actors, the observations, the workflows, etc

– Proposals (types)– Targets (observation modes, scientific

requirements, breakpoints)– The OBs and SBs– The CTAO workflow

→ First definition of the Data Model of the Proposal Handling

Insertions within the CTA General Data Flow

• Formal architecture approach towards definition of the systems, functionalities and interfaces

→ Definition of the interfaces for the Proposal application

The APC Proposal Handling Platform : A prototype for CTA

From Use Cases to Functionalities

User

Archive

Observation Data ProcessingProposal Evaluation Data Release

CTA TAC

Follow Up

ConsultSubmit

Proposal

The web-application Tabs

• Possible sequences of the evaluation of GO proposals– Step 0 : Technical Review– Step 1 : Scientific evaluation by sub-panels– Step 2 : Ranking of target by panels – Step 3 : Final ranking by the large panel

Preparation Submission Evaluation Follow-up Admin

• CTAO doc.• PHP appli doc.• Calculators

• Fields to fill• Scientific

Justification to upload

• Access rights to sub-pannels

• Building of the accepted prop.

• Provide all infos to PIs

• Software and doc. Maintenance

• DB evolution and maintenance

• User and access rights mgt.

Internal TRC

International TAC

A lab project for a web-based application

• Internal development

• Internal development

• Project under the supervision for CTA consortium– And soon, following the recommendation of CTAO (from a

s/w project manager)

– Started in 2013 in APC, first meeting within CTA in Feb. 2014– Survey of existing tools

● 9/10 July 2014 CTA/ESO workshop (Proposal Handling, Scheduler)● Either private, or not flexible enough, or based on 90's

technology

– With a software engineer and myself, development made within an apprenticeship training of an engineer school

– Using standard s/w project management (as used as for EUCLIDE)

→ Decision to make our own tool

Used technologies

CTA PHPWebportal

Proposal

Evaluation

Django Project ApplicationsWeb,

Database……

Project

• MariaDB Server• Database Python

Connector• Datamodel

• Update script• Generated

Documentation

Adm

in. P

rod.

To

ols

& Re

sour

ces

The CTA Science gateway based on Authorization & Authentication (A&A) system is used to identify a CTA user.

Forge :• forge.in2p3.fr/projects/cta_php

Repository :• https://gitlab.in2p3.fr/

IT Ressources: • 3 VM :

• Production server• Pre-production server• NFS server

• 2 Containers (Singularity): • Simulation• PDF Generation

Our s/w add-ons

• The used Data Model is the one of this CTA building phase

• But how stable will be over >20y?

• Development of “Modularity”– Over the Data Model– Over the Front-End

– Based on the CTA Top Level Use Cases

– Issue of maintainability (cost)– Issue of backward compatibility

→ The DB structures and the fields of the web forms are not hard-coded, just defined through a simple web interface

1. Design/create Modularity classes [done in the PHP]2. Describe classes without coding, add attributes/options, manage validity… [by an admin, when

implementation of the Data Model]3. Generate classes/models in Django/Python language [Function of the PHP]4. Generate Modular Classes in Database with Implementation’s Engine [Function in the PHP]5. Data model up to date: Dynamic UI according to classes’ description

Our s/w add-ons : the Modularity

Configurationof classes

ObjetsClasses

AttributesChoice_List

Generator of classes

Proposal classes :

ProposalProposal_Submit

TargetTarget_Submit

Engine of implementation

User « persistant »

Classes

ModularityclassesCta_Class

Field_DefinitionChoice_List

PersistantClasses

1

2 3 4

5

Our s/w add-ons : screenshot for the class confguration

Some screenshots

Some screenshots

Conclusions and Discussions

Main features

• Development of our own Proposal Handling Platform for CTAO– Will be proposed for CTAO as

in-kind contribution– Using modern s/w languages

for a web-based application– Permit the management of the

future Data Model evolutions by a simple 'secretary'

Asked features

• Current Proposal Submission System– CTA has not yet decided on a specific proposal submission system– Interfaces, functionalities, architecture, requirements currently being revised– Current prototype is web-based– Why web-based?

● Portability across platforms● Single access point for all users● Embedded within a common and uniform CTA science gateway

• Observatory Challenge – challenges specific to our observatory– Open and proposal-driven observatory → world-wide submission of

proposals– Three types of telescopes– Support for different sub-array configurations– Support for different Observing modes– Different environmental conditions: best quality for spectral analyses, less

favourable conditions for detection only– Need to support users during proposal submission with proposal support

tools to select best options for the proposal

Asked features (2)

• Observatory Challenge – proposal traffic– World-wide submission, esp. during last days of the AO calls

● What is the expected number of users per day?● What is the expected number of users per min in the last day/hour?

• User Database– How many registered users

● What is the expected number of users world-wide during operation phase ?

● (already 965 physicists just within the Consortium)– Authentication:

● Currently LDAP● Advanced authentication/authorization systems are being evaluated (e.g.

eduGAIN), but not yet decided– User informations:

● Currently: first and last names, user name, uid, email, organisation (currently)

● Not yet defined for the operations

Asked features (3)

• Phase 1 and 2 : link between proposals and Observation Blocks– Each OB is link to n Proposals (1-n relationship)

• Future plans– Prototype work planned to continue– No final choice on final proposal system has been taken yet, CTA is

open for collaboration

• Problems and Issues– Currently in prototyping phase– Evaluation not yet started