EBI is an Outstation of the European Molecular Biology Laboratory. Publishing Web Services – a provider’s point of view Rodrigo Lopez External Services.

EBI is an Outstation of the European Molecular Biology Laboratory.

Publishing Web Services – a provider’s point of view

Rodrigo LopezExternal Services - EMBL-EBI

EMBRACE Workshop DTU June 2010

Outline

• 10 TIPS• BioCatalogue

• Anatomy of a service in the BioCatalogue

• Acknowledgements

EMBRACE Workshop DTU June 2-4 201021/04/232

Tip 1

• Plan• Identify and document the science• Identify the users• Search and document equivalent or similar services• How much will it cost?

• Breakdown IT and HR resources

EMBRACE Workshop DTU June 2-4 201021/04/233

Tip 2

• Characterise IT capabilities• Select sustainable infrastructure

• Never go ‘alone’• Get your IT team involved

• Select up-to-date technology• Your IT team should provide SANE/LATEST• Consider Virtualisation• Some fun: If the service is discreet, consider deploying in the

Cloud (EC2)?• What about Google, Azure, Soho, Yahoo, Salesforce,

Rackspace and the other clouds?

EMBRACE Workshop DTU June 2-4 201021/04/234

EMBRACE Workshop DTU June 2-4 201021/04/235

InternetInternet

Web node Web node Web node Web node Web node Web node

Zeus LB1 Zeus LB2

EMBRACE Workshop DTU June 2-4 201021/04/236

OY PG

HX

InternetInternet

Tip 3

• Identify HR capabilities• Your programmer’s capability

• Qualifications• Discipline (adopt agile project management?)• Academic vs professional aptitudes• Is (s)he a good scientist?

• Don’t turn a good scientist into a web master!• (or into a nerd)• (or into a geek)

• Programming languages• Spring, Apache Axis2, CFX...• PERL, PYTHON, PHP, Java, C/C++/C#...

• Make choices according to the footprint of your IT/HR.

EMBRACE Workshop DTU June 2-4 201021/04/237

Tip 4

• Define the service architecture constraints• Modular or integrated architecture (monolithic or diversified)

• Small components can play many roles• Many services have dependencies!

• Integrated• Usually few people understand the complexity of the

application.• SOA definitions that apply to your service• Specifications and Organisations

• Read/Adhere to the EMBRACE recommendations• BioXSD (don’t invent unless you have to!)

• Get your XML background, specifications and vocabularies (EDAM), right.

EMBRACE Workshop DTU June 2-4 201021/04/238

Tip 5

• Define application server tier• Security & LoS

• Make sure you are safe• Don’t advertise security, unless you have to.

• Do you have a LoS (e.g. 98%, 24/7, etc.)?• State maintenance

• Is your application stateless or stateful? • Data access

• Is your back-end up to scratch• Optimise connection pools

• Persistence• Do you need it?

• Monitor the service• ...and profile it regularly (at each data update?).

EMBRACE Workshop DTU June 2-4 201021/04/239

Tip 6

• Service Redundancy• 2 nodes (physical/virtual)

• Round Robin, Simple Fail Over, Dynamic Round Robin, Dynamic Fail Over, Load Balanced?

• Service failover• Local• Remote (federated?)

• Recovery policies?• Maintenance policies?

EMBRACE Workshop DTU June 2-4 201021/04/2310

Tip 7

• Adhere to existing data delivery standards• Formats, styles...

• Adhere to existing data consumption standards• Results integration (BioXSD)

• Ensure integration into existing workflows or pipe-lines

EMBRACE Workshop DTU June 2-4 201021/04/2311

Tip 8

• Ensure your services are easy to understand and consume

• User support• 1st-3rd level support• Clarify Responsibilities

• Documentation• Programmer• End-user

• Build and publish example clients• Test in as many programming languages as possible!• Test with as many WS Enactors as possible!• Test, test, test...

EMBRACE Workshop DTU June 2-4 201021/04/2312

Tip 9

• Justification• Keep logs• Keep user communications• Document costs• Usage trend analysis

• Know your three A’s• Accessibility• Accountability• Awareness

EMBRACE Workshop DTU June 2-4 201021/04/2313

Tip 10

EMBRACE Workshop DTU June 2-4 201021/04/2314

•Publish your service!

Conclusions 1/2

• Identify your services life-cycle

EMBRACE Workshop DTU June 2-4 201021/04/2315

Conclusions 2/2

• Implement (if you have not already done so) Business Intelligence into your services operation• Make a INVENTORY of all your services’ components• Don’t forget to include all variables!• Keep it up-to-date (or get someone to do this for you)

• So, you have an INVENTORY (and you huff and you puff...)• Use it to market your services...

EMBRACE Workshop DTU June 2-4 201021/04/2316

BioCatalogue

• A means to pool metadata about services in the wild• A means to discover and reuse those services• A means to curate services• A platform for service monitoring and analytics

EMBRACE Workshop DTU June 2-4 201021/04/2317

BioCatalogue Annotation ‘guidelines’

• Where is the documentation?• What is it about?• Can anyone use it?• Who to contact?• What does the operation do or what are the

resources?• What is the input?• What is the output?

EMBRACE Workshop DTU June 2-4 201021/04/2318

BioCatalogue

• Anatomy of a service in the BioCatalogue

• http://www.biocatalogue.org/

EMBRACE Workshop DTU June 2-4 201021/04/2319

EMBRACE Workshop DTU June 2-4 201021/04/2320

Looking for a rigorous sequence search service...

EMBRACE Workshop DTU June 2-4 201021/04/2321

EMBRACE Workshop DTU June 2-4 201021/04/2322

EMBRACE Workshop DTU June 2-4 201021/04/2323

Ownership/Responsability

EMBRACE Workshop DTU June 2-4 201021/04/2324

Methods

EMBRACE Workshop DTU June 2-4 201021/04/2325

Monitoring

EMBRACE Workshop DTU June 2-4 201021/04/2326

EMBRACE Workshop DTU June 2-4 201021/04/2327

EMBRACE Workshop DTU June 2-4 201021/04/2328

BioCatalogue - Usage: Wider Context

EMBRACE Workshop DTU June 2-4 201021/04/2329

• Partnerships:

• Collaborations:

How to contact us:

• Please use:

• http://www.biocatalogue.org/contact• http://www.ebi.ac.uk/support/

EMBRACE Workshop DTU June 2-4 201021/04/2330

Acknowledgements

• EMBL-EBI/ES

• EMBRACE

• BBSRC

• EMBL

• NIH

• EPO

EMBRACE Workshop DTU June 2-4 201021/04/2331