APEX Experience from the Accelerators Controls and Operations Data Management in the Beams Department Antonio Romero Marin, Aurelien Fernandes, Jose Rolland Lopez De Coca, Nikolay Tsvetkov, Zereyakob Makonnen, Zory Zaharieva BE-CO Database Developers Forum – APEX 20.05.2014
73
Embed
Antonio Romero Marin, Aurelien Fernandes, Jose Rolland Lopez De Coca, Nikolay Tsvetkov, Zereyakob Makonnen, Zory Zaharieva BE-CO Database Developers Forum.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
APEX Experience from the Accelerators Controls and Operations
Data Management in the Beams Department
Antonio Romero Marin, Aurelien Fernandes, Jose Rolland Lopez De Coca, Nikolay Tsvetkov, Zereyakob Makonnen,
Zory Zaharieva
BE-CO
Database Developers Forum – APEX
20.05.2014
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 22
Contents
• Introduction to the Controls environment and some of the services
using APEX
• Interesting features for developers
• APEX_COMMON Framework
• Plugins & custom components, look and feel features, etc.
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 3
Introduction to the Accelerators Data Management
• Complex and challenging area
• Federation of various databases, e.g.• Accelerators Controls Configuration• Diagnostics and Monitoring (DIAMON)• Alarms (LASER)• Accelerators Entities and Signals Naming• Logging and Measurements, etc.
• Strategy• Relational DB technology – Oracle DBs for all technical data• Data-driven applications and APIs - Oracle technology stack
• Reliable database services and user interfaces (24/7/365)• On-line usage of database services for the accelerators Control and
Operation
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 4
APEX Development throughout the years
• Starting with HTMLDB• beginning of 2005, HTMLDB v.1.6
• Need for rapid application development (Agile programming)• fast prototyping and short time to production deployment
• quickly respond to new user requirements
• Ready to use templates, widgets and page components• professional look-and-feel with minimum effort on design – concentrate on the
application itself
• The nature of the interfaces to be implemented• web-interfaces, database-centric• APEX supports client-side scripting for more dynamic / interactive / feature-rich
interfaces
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 5
First Trials of APEX
• Data browsing interfaces and portal for the Accelerators Entities and Signals Naming Database
• The Naming DB – dictionary for the accelerators complexes• Equipment types• Equipment positions in the accelerators• Settings and observables (signals)
• Read-only interfaces to the data• LHC Equipment catalogue• SPS equipment catalogue• PS equipment and layout components catalogue• Signals Simple Extraction Interface• Signals Experts Extraction Interface• Naming Domain Portal
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 6
Accelerators Naming DB Interfaces
• Data-browsing Interfaces (window-on-data) – development from scratch• low level of complexity• no transactions• some business logic to get the data out• user authentication and simple user authorization• easy extraction of data into XML/CSV files
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 7
Portal Pages and News feeds
• Simple portal pages
• Access to other related interfaces, documentation, help pages
• Dynamically constructed using data stored in tables
• News, RSS feeds, News Archive, etc.
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 8
History Log Browsers
• History logging in our dbs• All data modifications are recorded (to know who did what and when)
• History Log Browser objectives• Access the history log for any table in the database for which history is
enabled • Provide simple search capabilities for the history logs• Provide a summary for the changes of a given record based on the PK
• Completely dynamic configuration• List of tables, list of columns per table, primary keys for tables (USER_
tables)• Dynamic PL/SQL embedded in APEX regions
• Portable solution• Deployed to several of the databases, which are using the same mechanism
for keeping history without any changes to the interface components
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 9
History Log Browsers
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 10
• The heart of the Controls System – the basis for the Configuration Management of all Controls System components in the entire accelerator complex
• Provides data for all configuration items and their relationships, required for the correct functioning of the Controls System
• All controls devices (>105,000) and parameters (> 2,000,000)
• Hardware and software configuration of all Front-End Computers
(>3,500), Drivers generation
• The Accelerators Timing System, Operators Consoles, etc.
• Service with 30 years of history• Supporting the requirements of the entire accelerator complex – LHC, SPS and PS
complex
Controls Configuration Service
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 11
Controls Configuration Data Browser
• An existing PL/SQL OWA application with ~ 100 reports• existing PL/SQL code base in the CCDB• a complete re-engineering of the application was undertaken in 2006• possibility to reuse some of the PL/SQL code• ‘thick’ database layer – PL/SQL packages in the database
• The first big scale data browsing application to use APEX
• Nowadays >220 reports used by a user community of >300 people - thanks to• the ease of APEX reports development• Introduction of APEX user interactive reports
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 12
Alarms (LASER) Database
• LASER (Alarms) - capturing, storing and notification of anomalies for the whole accelerator chain and technical infrastructure
• Alarms DB - 3 different database areas• Pre-defined alarm definitions• User configurations for the alarms displays• Time-stamped run-time alarms events
• First APEX data editing interfaces - 2010• fast development• evolution of APEX• suite of Data Management Tools (7 Editors)• developed from scratch• give users the possibility to explore their
data and maintain it
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 13
Controls Configuration Data Editors
• Data-manipulation interfaces• complexity of the applications due to transaction management• complex business logic to handle modifications of the data• stringent data access rights – e.g. specific authorization modules, virtual
private db, etc.
• Existing web-deployed, rich clients, using Oracle ADF
• Renovation of the existing interfaces and providing new ones with APEX since beginning of 2011
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 14
Controls Configuration Data Editors
• The first relatively big scale and complex editing applications to be developed / migrated to APEX
• So far - 15 Data Editors (> 100 forms), ~ 250 user community
• 4 editors remaining to be renovated
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 15
Internal developers applications
• Due to the ease of development, the developers are using APEX to create applications for managing some of the developers specific processes
• Admin Editor for Service Interventions
• DB Schema Change Management Editor
• APEX _COMMON Metadata Editor
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 16
APEX in the Controls Environment nowadays
• Specific reporting tools• used for CCS, Laser, Diamon, Accelerators Naming, Measurements • > 300 reports
• Generic reporting modules and applications for managing the developers tasks• History Browser; Schema Change Management; Admin Interventions Config, etc.
• Portals and news pages with RSS feeds
• Data editors• used for CCS, Laser, Diamon, Layout, R2E, SM18 Data Analysis • > 25 editors (> 200 editing forms)
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 17
Using APEX for the accelerators data management
• Flexible environment for development• Allows you to do a lot of things• It is up to the developers to provide a structured approach to application
design
• Complete integration with our databases (Oracle, PL/SQL)
• Provides a lot of features and components ‘out of the box’• Fast and easy development
• We can do even better • Custom libraries and components
• Best practices, guidelines and coding standards
Speed and Scalability
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 1818
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 45
Plugins – usage and observations about them
• The most interesting and often used ones• Custom pop-up plugins for selection• Save before exit plugin
• Advantages of the custom pop-up plugins• possibility of quick search in multiple columns• shows structured data based on a custom SQL query• possibility of multiple columns update • doesn’t open new browser window
• Find “Custom Pop-up plugin for selection” Wiki pagehttps://wikis.cern.ch/pages/viewpage.action?pageId=70681028
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 49
Save before exit plugin
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 50
Save before exit
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 51
Save before exit
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 52
• What was the challenge?• columns generated dynamically based on data stored in
a table
• How do we do it?• build SQL dynamically using pivot in order to transform
data stored into table rows and map them to columns • use “Generic Column Names” tabular form option• save the data in the DB using unpivot
Advanced manual tabular form with dynamic number of columns
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 53
Dynamically generated columns in a tabular form
Table1 Devices – names of devices stored in rows
Table2 LogicalEvents – stored in rows and presented in columns.Devices from the same class have the same number of LogicalEvents
Table3 Event Configuration – data stored in rows, based on the device and LogicalEvent
APEX Experience from the Accelerators Controls and Operations Data Management 20.05.2014 54
Bulk data loading features
• What is bulk data loading?
• Who benefits from providing this functionality?• users are able to import big sets of data using CSV files• support team reduces the number of requests
• We improved the standard APEX component• to provide more structured information• to add different highlighting for all the actions• to show result details for all updated and inserted rows
• Find the Wiki page on the following link https://wikis.cern.ch/display/config/CSV+data+import+-+APEX+Data+Loading+wizard