5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 1 ESRF Status Long Term Strategy Computing infrastructure Control systems
Jan 18, 2016
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting
1
ESRF Status
Long Term Strategy
Computing infrastructure
Control systems
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 2
Long term strategy• Nano science center• 10 120m long beamlines (down to 20nm beam size)
• Extreme condition science (5Mbar,50Teslas,5Kk)
• Enlarge exph by 8000m² + 10000m² labs• new detectors (larger, faster data, larger data)
• Topping-up operation ->new injector design
• Higher intensity ->new absorbers,chambers,RF cav
• Canted undulators, new lattice, more beamlines
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 3
Long term strategy
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 4
LTS consequences on computing• Data rate x 1000 for new detectors
– Need new network infrastructure– Need new storage facility (Petabyte range)– Need new compute clusters (Petaflop…)– Grid computing, Parallel software development– Online data analysis becomes necessary– Virtualization of data storage, databank with
metadata.– New 500m² data center
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 5
LTS roadmap
• Presented at council spring 2006
• Detailed study report is spring 2007– Detailed technical analysis– Costs and planning– Human resources
• First phase starting in spring 2008
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 6
Experiment data statistics
• 2005 summary– 197 TBytes generated– 62 Millions of files
• 2006 evaluation– 280 TBytes (1TB/day!)– 100 Millions of files
• Present storage facility about 100TB
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 7
Topology…putting it togetherTopology…putting it together
Q3156kx6k pixels35 MB/s
Beamlines Central BuildingComputer Room150 m2, 75 kW
CTRM Computer Room150 m2, 75 kW
Network AttachedRAID Storage
Compute Clusters
1 Gbps 10 Gbps
Internet
1 Gbps
Frelon2kx2k pixels8-32 MB/s
1 Gbp
s
4 Gbps
Tape Robot
FC attachments
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 8
Comments on data statistics
• x 10 since 2001• We expect x 10 in the next few
years (10 TB/days …)• May be 99% of 0 and 1% of real
data.• Need to do online analysis!
– Eliminate wrong data, extract the information as early as possible.
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 9
Control systems
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 10
Operating system convergence
PCI bridge
Suse 7.2
SolarisOS9
Suse 8.2
Debian30
60 VMEs
40 PCs
6 CompactPci
10 Workstations
10 PCs
Solaris servers
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 11
Libera devices @ ESRF
• 3 for Bpm for transfer line in operation• 3 for Bunch per Bunch feedback• Possibly many other in the future
– (224 SR + beamlines BPM + other projects…)
• Low quality ethernet interface– (weak, unreliable, non standard)
• Release policy unclear, technical and commercial support difficult.– SOLEIL, are you happy with it?
• We need to make pressure together!
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 12
TACO vs TANGO
• TACO– 8500 devices (defined)
– 170000 properties
• TANGO– 2000 devices (defined)
– 18000 device properties
– 500 server instances
– 97 hosts (67 on events)
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 13
HDB architecture
TACO HDB
Filler
DC cache
Taco servers
Polling
HDB_query
Archive eventhandlerTango servers
Archive eventsConfig server
HDB_config
TANGOHDB config
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 14
SrGauges
VacCellGaugeVacCellGaugeVacCellGauge
TangoScalar
Attributes
TangoSpectrumAttribute
s
History
500 Devices
90 Servers
32 Servers
VacGauge
PgBalzers
GaugeGauge
Serialline
TACOcommand
GaugeGauge
VacGauge
PgBalzers
GaugeGauge
Serialline
TACOcommand
GaugeGauge
VacGauge
PgBalzers
GaugeGauge
Serialline
TACOcommand
GaugeGauge
VacGauge
VIP
GaugeGauge
Serialline
TACOcommand
GaugeGauge
VacGauge
VIP
GaugeGauge
Serialline
TACOcommand
GaugeGauge
VacGauge
VIP
GaugeGauge
Serialline
TACOcommand
GaugeGauge
Vacuum system
To HDB
5 6 september 2006 JM Chaize, ESRF TANGO collaboration meeting 15
Pb of dynamic attributes
• Residual Gas Analyzer (100 dynamic attributes)– Potentially 64 devices x 100 attributes
• Thermocouple controller (64 dynamic attributes) fully dynamic!
• Need automatic way to configure storage