7 Sep 2006NVO Summer School 20061 T HE US N ATIONAL V IRTUAL O BSERVATORY Accessing Web Services Matthew J. Graham CACR/Caltech.

7 Sep 2006NVO Summer School 2006 1

THE US NATIONAL VIRTUAL OBSERVATORY

Accessing Web Services

Matthew J. GrahamCACR/Caltech

7 Sep 2006NVO Summer School 2006 2

Overview

• Definition of a web service• Web service types• Underlying technologies• Usage models• Federation• “2.0”

7 Sep 2006NVO Summer School 2006 3

Web service deoccultation

• Invocations• Strange languages• Action at a distance• High priesthood

7 Sep 2006NVO Summer School 2006 4

What is a web service?

• “A software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine-processable format.” - W3C

WSDL and SOAP conveyed using HTTP with an XML serialization + other Web-related standards

• Not a new idea: – RPC/RMI – CORBA– DCOM– XML-RPC

7 Sep 2006NVO Summer School 2006 5

Web service models - I

• Resource oriented model– A resource is anything that can have an

identifier (URI)– Focuses on resource description and

representation (data object reflecting state of resource)

• Service oriented model– A service is realised by an agent (provider) and

used by another (requester)– Focuses on tasks (unit of action) that may be

performed by an agent: description, messaging/choreography and goal state

7 Sep 2006NVO Summer School 2006 6

Web service models - II

• Message oriented model– A message is the basic unit of data sent from one

agent to another: the action taken in response to receiving a message is up to the recipient

– Focuses on message structure (contract) and transport (choice of protocol carries no semantics)

• Policy oriented model– A policy is a set of assertions expressing

capabilities and constraints– Focuses on security, quality of service and

management

7 Sep 2006NVO Summer School 2006 7

How’s it done in the real world?

• “Things should be made as simple as possible, but no simpler” - Albert Einstein

• WWW is the largest, most distributed and scalable application on the planet:– Objects (resources) are identified by URIs– Resource state information (representations) is

exchanged in many data formats – Protocols that support interaction between

agents and resources - choice of protocol places limits on which representation formats can be transmitted HTTP and HTML (XML)

7 Sep 2006NVO Summer School 2006 8

REST: The formal approach

• Architectural style not an implementation(http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm)

• Each resource has a URI• Exchange resource representations

(XML)• Uniform interface semantics (CRUD

API):

HTTP protocol CRUD action Description

POST CREATE Create a new resource

GET RETRIEVE Retrieve a resource representation

PUT UPDATE Update a resource

DELETE DELETE Delete a resource

7 Sep 2006NVO Summer School 2006 9

Accidentally RESTful

• HTTP GET/POST + XML (POX/HTTP)• Verbs allowed in URIs• Requires little new infrastructure - just HTTP and

XML processing technologies• Simple clients, e.g. wget or xsltproc• Commercially popular (85% of traffic, 6x faster):

– Amazon (http://www.amazon.com/gp/aws/landing.html)– Yahoo (http://developer.yahoo.net)– eBay (http://developer.ebay.com/rest)– Flickr (http://www.flickr.com/services/)– YouTube (http://www.youtube.com/api2_rest)– Del.icio.us (http://del.icio.us/doc/api)

7 Sep 2006NVO Summer School 2006 10

AJAX (Asynchronous Javascript + XML)

• Uses browser’s XML support: DOM, XSLT• XMLHttpRequest• Google Maps is best-known AJAX application

7 Sep 2006NVO Summer School 2006 11

Other data formats - I

• JSON (JavaScript Object Notation, RFC-4627)– Subset of JS object literal notation (does not

require JS)– Data types: number, string, boolean, array, object,

null– Supported by many languages– In Dec 2005, Yahoo! added support for JSON

but– No schema mechanism (validation, code

generation)– Limited type system (no date or time)– No extension or versioning

7 Sep 2006NVO Summer School 2006 12

Other data formats - II

• YAML (YAML Ain’t Markup Language)– All data can be represented by lists, hashes and scalars– Superset of JSON: validate with Kwalify– De facto serialization format in Ruby; support in many

languages

• Microformats– Place marked up data in (X)HTML pages– Use HTML attributes: class, rel, and rev

• RSS/ATOM (RFC-4287)– Time-stamped uniquely-identified data chunks with metadata– Lucene Web Service API (http://dev.lucene-ws.net/wiki/API)– Google Data (GData) API

(http://code.google.com/apis/gdata/index.html)

7 Sep 2006NVO Summer School 2006 13

What do RESTful services lack?

• Format method for describing interface contract

• Reliable messaging• Digital signatures• Message routing• Resource life cycle management• Asynchronous event notification• Other capabilities captured by WS-* specs

7 Sep 2006NVO Summer School 2006 14

What is SOAP?

• Simple Object/Service-Oriented Access Protocol (Snakes On A Plane?)

• An XML-based communication protocol and encoding format for exchanging structured information in a decentralized, distributed environment

• W3C specification (http://www.w3.org/TR/soap)

7 Sep 2006NVO Summer School 2006 15

Anatomy of a SOAP message

• An envelope to encapsulate data which defines formatting conventions for describing the message contents and routing directions: header and body

• A message exchange pattern: request/response (RPC mechanism), fire-and-forget

• A transport or binding protocol• Data encoding rules for describing the

mapping of application-defined datatypes into an XML tag-based representation

7 Sep 2006NVO Summer School 2006 16

SOAP example

Request:<soap:Envelope xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xsd=http

://www.w3.org/2001/XMLSchema xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body>

<ComovingLineOfSight xmlns="http://skyservice.pha.jhu.edu"> <z>float</z> <hubble>float</hubble> <omega>float</omega> <lambda>float</lambda>

</ComovingLineOfSight> </soap:Body>

</soap:Envelope>

Response:<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body>

<ComovingLineOfSightResponse xmlns="http://skyservice.pha.jhu.edu">

<ComovingLineOfSightResult>float</ComovingLineOfSightResult>

</ComovingLineOfSightResponse></soap:Body>

</soap:Envelope>

7 Sep 2006NVO Summer School 2006 17

Client Invocation Models

• Static: use generated stubs:java org.apache.axis.wsdl.WSDL2Java <wsdl url>

• Dynamic:– no generated code– a proxy dynamically generates a class at runtime that

conforms to a particular interface, proxying all invocations to a single ‘generic’ method

– Examples: • Java : use javax.xml.rpc.Service.getPort() and createCall()• .NET : use RealProxy class (must extend ContextBound) or

Reflection.Emit

• Generic SOAP client: http://soapclient.com/soaptest.html

7 Sep 2006NVO Summer School 2006 18

Why is SOAP better?

• Asynchrony• Routing• Reliable messaging (e.g. once-and-only

delivery, guaranteed or exact execution)• Send and receive complex datatypes to invoke

a particular method not just key-value pairs • Security • Binds to other protocols• Service description

7 Sep 2006NVO Summer School 2006 19

Mashosphere: federating web services

• Mashup styles:– Presentation– Client-side data– Client-side software

• Mashup technologies (“Rich Internet Applications”):– Laszlo (http://openlaszlo.org)– Xforms (http://www.w3.org/MarkUp/Forms)– XUL (http://www.mozilla.org/projects/xul)– Dojo (http://dojotoolkit.org)– ATF (Open Ajax Alliance)

– Server-side software– Server-side data

7 Sep 2006NVO Summer School 2006 20

SOA and WOA

• An application architecture within which all functions are defined using a description language as independent services with invokable interfaces which can be called in defined sequences to form business processes.

• Principles:– Service reusability– Service contract– Service loose coupling– Service abstraction

• Virtual Observatory = Service Oriented Astronomy• WOA advocates REST or POX over HTTP

– Service composability– Service autonomy– Service statelessness– Service discoverability

7 Sep 2006NVO Summer School 2006 21

WOA/Client

• Inclusive: web services means anything over HTTP• Agnostic and Consistent: platform, technology,

protocol, data format, contract description language, programming language

• Expect Constant Change: contract, protocol, endpoint

• Mashup-Oriented: integrating data from many sources and re-serving it

• Deeply Resilient: extreme fault tolerance, mandatory pessimism about quality of run-time environment (Web)

• Bare Data: transformation costs, lack of control• Extreme Loose Coupling