Jon Udell, InfoWorld Practical Aspects of Web Services 1995 to 2005
May 11, 2015
Jon Udell, InfoWorld
Practical Aspects of Web Services
1995 to 2005
Jon Udell, InfoWorld
From LAN to WAN
• 1995 – present: The services Web is like a big slow local area network– Dominant metaphors: client/server, remote
procedure call
• present – 2005: The services Web becomes more like an internetwork– Dominant metaphors: network of peers, document
routing
• Continuous evolution, no dramatic tipping point
Jon Udell, InfoWorld
The services Web as a LAN
• 100 milliseconds not too different from 2 milliseconds
• Internet apps always took client/server for granted (telnet www.infoworld.com 80)
• HTML/JavaScript is a great universal client, good enough (still) for lots of things
• CGI is a great universal service delivery model, also good enough for many things
• HTTP the universal transport– Port 80 is “open for business”
Jon Udell, InfoWorld
Proto-web services (1995)
• From BYTE.com I link to a CGI service on another site.
• That site hosts the form, and also the script behind the form.
• I need to control the form myself.• Can we decouple these things? Yes! It’s
trivial. • Distributed services were built in from the
start, so deeply that we couldn’t even see it.
Jon Udell, InfoWorld
Proto-web services (1996)
• BYTE’s parent, McGraw-Hill, runs a federation of websites.
• Federating search across the sites looks like a multi-man-year, multi-thousand-dollar project.
• But…AltaVista already indexes those sites. • And AltaVista is a website.• Therefore it is a web service, by (my) definition.• A Perl wrapper around AltaVista proves that
federated search was really a four-hour, zero-dollar project.
Jon Udell, InfoWorld
Proto-web services (1998)
• Yahoo offers a web service that organizes links by category
• AltaVista offers a web service that counts references to pages in its index
• These components can be joined to create a new web service that measures web mindshare
• http://udell.roninhouse.com/mindshare-report.html
• You didn’t need a Google-style SOAP API to do this (but it’s sure nice to have one!)
Jon Udell, InfoWorld
Building and using proto-web services
• Regex-capable, URL-aware scripting languages: Perl, Python, Ruby– Downplayed in the Java/.NET era, but still (I claim)
crucial
• Reverse-engineering Web APIs– Local proxies tap the Web’s pipelined architecture– I use Proxomitron – It even supports SSL– These things will remain useful for a while. Maybe
a long while. It’s all about integration.
Jon Udell, InfoWorld
What proto-web services got right
• Statelessness (massive scale-up)– In 1994 I had a dozen users. By 1996, ten
thousand a day.
• Simple, standard addressability: every resource has an address
• Same interfaces for humans and for software
• Protocol messages and payload messages easily accessible
Jon Udell, InfoWorld
What proto-web services got wrong
• Statelessness– Not an unmixed blessing– Had to invent app servers to manage sessioning
with cookies or URL-threaded tokens– If reliable messaging needed, app layer has to
provide it – Always struggling to establish/maintain context
• Message and payload formats– readable not the same as mechanically parseable– service description hardwired to app presentation
Jon Udell, InfoWorld
Early hybrid solutions
• webMethods (1999) had a great idea– Automate the HTML screenscraping, simplifying
production of services. – Offer a clean and stable XML-over-HTTP API over
a messy and mutable HTML-over-HTTP API.– Point consumers at the stable APIs. – Anticipated a seamless transition to XML as the
primary interface, with HTML derived from it
• It’s still a great idea :-)
Jon Udell, InfoWorld
Dawn of XML-over-HTTP
• Dave Winer’s MailToTheFuture– Send yourself an email reminder– A core service that is presentation-neutral– Has an HTML presentation– And an XML-RPC API
• XML-RPC is the proto-SOAP– Data marshalling: strings, numbers, dates, and
structs– Many script-language bindings– Trivial cross-language interop was an eye-opener
Jon Udell, InfoWorld
Stupid web services tricks
• UserLand was aggregating XML (RSS) news feeds• OreillyNet wanted to receive the aggregated feed• Dave Winer wrote an XML-RPC client (Frontier/UserTalk) to transmit
it• I wrote an XML-RPC server (Linux/Apache/Python) to receive it• This was cool. Distributed computing, over the Net, between
UserTalk and Python• It was also dumb. RPC was overkill. We only really needed to fetch
XML files. • This is how all large-scale RSS aggregation works today• Moral: Just because you can use web services doesn’t mean that
you always should. • Corollary: Different definitions of web services appropriate to
different situations
Jon Udell, InfoWorld
REST and the power of the URL-line
• Representation State Transfer– Roy Fielding’s design rationale for the web– Everything has an address (URL)– URLs are human- and machine-invokable– Can be bookmarked, emailed, written into web
pages, and pipelined through URL-aware software
• SOAP vs REST a false dichotomy– SOAP toolkits can surface HTTP GET bindings– MS Visual Studio.NET and The Mind Electric’s
GLUE already do this– Where appropriate, WSDL support for HTTP GET
binding of SOAP calls can be and will be available
Jon Udell, InfoWorld
RESTful strategies• Many useful services return XML in response to
conventional HTTP GET or POST– RSS newsfeed conversion
• Macromedia offers a URL-based non-RSS XML feed• W3C offers an URL-based XSLT service• I post an XSLT stylesheet that converts the feed to RSS• A new URL-based service is formed by invoking the others in
combination• http://weblog.infoworld.com/udell/2002/05/19.html#a252
– Keynote’s new Data Pulse• Fetch measurements in realtime, as XML, with HTTP GET• Where’s the SOAP wrapper?• Why do you care?
• RESTful strategies have low coordination cost
Jon Udell, InfoWorld
XML gateway architectures
• By 2000, it was clear that a typical web-facing application should:– Express core logic and content as presentation-neutral XML– Derive HTML UI from XML (e.g., using XSLT)– Export component interfaces as XML-over-HTTP– Even before consensus around SOAP/WSDL, this was a
good strategy and some followed it– The hard work was, and is, to define the right XML-oriented
service and content models– That task is independent of your choice of SOAP toolkit, or
the current state of SOAP interop– Maybe even independent of SOAP– Example: Safari XML gateway
• SOAP or no SOAP, this still isn’t happening enough
Jon Udell, InfoWorld
SOAP sooner and later
• Sooner and later, SOAP does matter
• Now, for the World Wide LAN, an OS- and language-neutral RPC– Enormous network effects yet to be felt
• Later, for “application internetworking” -- a kind of IP protocol that routes messages (documents)
Jon Udell, InfoWorld
SOAP styles: RPC• Most first-generation SOAP toolkits use RPC
style– The SOAP envelope encodes a representation of a
method call with typed arguments– Or a response packet with structured/typed data– But XML is not a first-class datatype
• “DCOM or CORBA with angle brackets”– Corresponds nicely to programming tools and habits– OS and language neutrality a huge boon– Appropriate for the World Wide LAN, not the World
Wide WAN
Jon Udell, InfoWorld
What’s wrong with SOAP RPC?
• Endpoints are “tightly coupled” and interfaces are “brittle” – In general, but not necessarily or always– Good programmers will separate interfaces from
implementations
• Doesn’t leverage the versatility and self-descriptive nature of XML– RPC uses XML to represent strings, numbers,
structs: programming constructs– Pure XML is about documents (portable object
databases)
Jon Udell, InfoWorld
SOAP styles: document
• Endpoints are “loosely coupled” and interfaces are “flexible”– The SOAP envelope is just an XML document– XML is a first-class datatype– XML Schema is the data definition language
• Hailstorm API the most complete example– More a document exchange protocol than a
conventional RPC-style API– Inherently distributable– Presumes a persistent XML datastore
Jon Udell, InfoWorld
What’s right about document style?• Endpoints are “loosely coupled” and interfaces are
“flexible”– In general, but not necessarily or always– Bad programming can still defeat the intent to
separate interfaces from implementations• Leverages the versatility and self-descriptive nature
of XML– But more degrees of freedom mean more work for
programmers– Need an out-of-band signaling scheme to
communicate intentions
Jon Udell, InfoWorld
XML mapping separatesinterface from implementation
• Don’t derive WSDL from implementation namespace
• Instead, abstract a stable XML grammar• When implementation changes, SOAP
endpoints do not (necessarily) break• BEA’s WebLogic Workshop does this today• SOAP’s document style represents the
general solution
Jon Udell, InfoWorld
One-way SOAP messaging
• SOAP (since v 1.2) is intrinsically a one-way messaging protocol– Asynchronous messaging– Fire-and-forget, with callbacks and
message correlation
• Design patterns for one-way messaging– Federated search– Publish/subscribe (Kenamea, KnowNow)– SOAP routing
Jon Udell, InfoWorld
Why SOAP routing?• Microsoft is behind it
– GXA: Global XML Architecture• WS-Routing (was SOAP-R)• WS-Referral
• It’s the right thing (long-term)– Add value without modifying endpoints– Rohit Khare: “SOAP routing addresses layers 8 and 9 of the OSI stack:
political and economic”– A way to “reason about third- and fourth-party interests”
• Web services are political and economic– Scary, but inevitable– Example: a tax payment service is necessarily embedded in a regulatory
environment
Jon Udell, InfoWorld
First uses of SOAP routing
• Negotiating trust boundaries the main challenge
• As usual, first practical deployments will occur within the firewall
• Initial cross-firewall uses will rely on centralized, not distributed, trust– Web services hubs, like Grand Central
Networks, the practical near-term solution
Jon Udell, InfoWorld
Vision of the services Web
• From milliseconds to days/weeks/years• From client/server RPC to a routed message
fabric• HTML/JavaScript augmented by rich
clients: .NET, Java, Flash• CGI replaced by web services• HTTP augmented by ???
– Do we retire every port but 80 and 443?– Do we extend HTTP for loosely-coupled, asynch,
routed messaging (e.g. HTTPR)?
Jon Udell, InfoWorld
Practical strategies for the services Web
– Embrace the data-driven software model– Make XML a core competency, just like
SQL– Combine software atoms (components)
into software molecules (services)– Bite the PKI bullet
Jon Udell, InfoWorld
Data-driven software development
• MetraTech’s web-services-based billing engine– Integration more declarative than procedural– Instead of writing to APIs, “partners just have to
agree on sets of name/value pairs”– Data mappings, not APIs, the integration DNA– When standards settle, and are packaged into
toolkits, what’s left are data structures– You still need to do the analysis. Nobody can
automate that.
Jon Udell, InfoWorld
XML core competency
• Documents are the currency of Web services– And by the way, most of your data is in
documents, not an RDBMS
• XML is as fundamental to this generation of developers as SQL to the last
• XPath, XSLT are new core competencies
Jon Udell, InfoWorld
Legacy services: from atoms to molecules
• Web services are everywhere– Stored procedures, COM/COM+, Java classes all
map to SOAP
• You can expose these endpoints directly– But it usually doesn’t make sense. Wrong
granularity
• Now’s the perfect time to refactor– As you wrap your atomic services, use the
opportunity to combine them into molecular business objects
Jon Udell, InfoWorld
The day of PKI reckoning• Myth: XML standards will solve web services security• Reality: Wrapping angle brackets around signing and
encryption won’t make things easier- This stuff’s been around forever- We’ve never really learned to deal with it- How many of you sign or encrypt email? Why not?- Web services ups the ante: signing/encrypting parts of
documents, not just whole documents and end-to-end sessions
- Get comfortable deploying and using keys and certificates
Jon Udell, InfoWorld
Conclusions
• The first-generation services Web got things more right than wrong– Apple’s Sherlock 3: “web services for the
rest of us”
• Practical strategies– Inventory/refactor software assets– Bone up on XML, PKI – Dust off those rich-client skills