Managed by UT-Battelle for the Department of Energy Best Ever Archive Utility, Yet (BEAUtY) Kay Kasemir kasemirk@ornl.gov April 2013.

Managed by UT-Battellefor the Department of Energy

Best Ever Archive Utility, Yet (BEAUtY)

Kay Kasemir

kasemirk@ornl.gov

April 2013

2 Managed by UT-Battellefor the Department of Energy

CSS-based OPICSS-based OPI

ArchiveArchive EngineArchiveArchive Engineconfig.xmlconfig.xml

IOCIOC

Binary Data Files

Data ServerData Server

“ASCII” Config.“ASCII” Config.Channel AccessChannel Access

XML-RPCXML-RPC

Previous Setup with Channel Archiver

History~2000:

Started byBob Dalesio

~2003:Index Tools,Data Server

~2007:CSS Client

3 Managed by UT-Battellefor the Department of Energy

Details Older Java “archive viewer” also available.

Archive Engines running on “sampling” computer, data server on “serving” machine.

Scripts to restart engines, copy data, update indices.

Headaches with data maintenance:– Index time grows with data

– Stuck when index files reach 2GB

– SNS Users faced with ~80 sub-archives

– No clue what needs fixing after network/power problems

– No idea who contributes how many samples

– No way to remove selected channels or time ranges

– Improving on this means implementing an RDB

4 Managed by UT-Battellefor the Department of Energy

CSS-based OPICSS-based OPI

ArchiveEngineArchiveEngine

config.xmlconfig.xml

IOCIOC

RDB(Oracle/MySQL)

RDB(Oracle/MySQL)

Channel AccessChannel Access

New Setup

SamplesSamplesConfig.Config.

EngineConfig-Import

Other tools for config & samples

5 Managed by UT-Battellefor the Department of Energy

RDB Alternatives

Data Browser– read from old Channel Archiver’s

ArchiveDataServer.cgi

– Read from RDB

– use your own site-specific ‘reader’ plugins

Archive Engine– Pluggable Configuration

– Pluggable ‘Write’ support

6 Managed by UT-Battellefor the Department of Energy

CSS Data Browser Handles Both New URL

Just one‘RDB’sub-archive

Old and newdata can becombined inone plot

7 Managed by UT-Battellefor the Department of Energy

View Archive Data in CSS Data Browser

Open Data Browser Plot

Add PV– If default archive setup lists data source with historic

data for the PV, it will be displayed

8 Managed by UT-Battellefor the Department of Energy

Searching for PVs in Archive

Open Archive Search View– Data Browser Perspective,

or Plot’s context menu

Search for channelsvia pattern

Send to new Data Browser– Via context menu

Send toexisting Plot– Via drag

& drop

9 Managed by UT-Battellefor the Department of Energy

Configure Sample Engines

Configuration is in RDB– Directly use SQL

– EngineConfigImport for legacy config files

– View/Edit via web

Hierarchical (as before)– Sampling engine (name, where to run, …)

Groups– Channels

No more duplicate channels!

10 Managed by UT-Battellefor the Department of Energy

Web Interface to Engine Config in RDB

Tomcat/JSP/Servlets to view and edit

11 Managed by UT-Battellefor the Department of Energy

Web Config View: Channel Stats

12 Managed by UT-Battellefor the Department of Energy

Web Config View: Sample Stats

OK

??

13 Managed by UT-Battellefor the Department of Energy

Stats 34 sample engines, 79400 channels

Host that runs sampling engine:– CPU load 45%, zero disk I/O wait, very responsive

Oracle performance– 8000 samples/sec peak in write tests

– Operationally maybe ¼ of that

Better configurationwould likely havefewer samples/sec

14 Managed by UT-Battellefor the Department of Energy

RDB Supported

Currently supported: MySQL or Oracle– Main table difference: Oracle TIMESTAMP has

nanosecs, MySQL has separate nanosecs col.

– All tools auto-configure based on JDBC URL

PostgreSQL support from ITER

15 Managed by UT-Battellefor the Department of Energy

Which one to use? Channel Archiver

– Fast

– Can’t be maintained

RDB– Slow

– Super easy to maintain

Worth a look: – SLAC Archiver Appliance

http://epicsarchiverap.sourceforge.net/

– HyperArchiver http://www.lnl.infn.it/~epics/joomla/archiver.html

– Both of them work with CSS Data Browser

16 Managed by UT-Battellefor the Department of Energy

Thanks!