Pragmatic XML By Peter Kriens, CEO aQute .

PragmaticPragmatic XML XML

ByByPeter Kriens, CEO aQutePeter Kriens, CEO aQute

www.aQute.swww.aQute.see

www.aQute.sewww.aQute.se - 2002 XML Portland - - 2002 XML Portland - 22

ContentsContents

The ProblemThe Problem TechnologiesTechnologies CoordinationCoordination ProductsProducts ConclusionsConclusions


The ProblemThe Problem

The OOPSLA (Object Oriented The OOPSLA (Object Oriented Programming Systems Languages and Programming Systems Languages and Applications) conference is held yearly Applications) conference is held yearly since 1986since 1986

Organization consists of a professional Organization consists of a professional office and many volunteersoffice and many volunteers

There are many different publicationsThere are many different publications– Advance and Final programAdvance and Final program– Web site, PDAWeb site, PDA



Committee

OOPSLAOffice

Exhibitions

Participants

CommitteeSub

Committee

Participants

ExhibitorExhibitor

Local info

FinalProgram

AdvanceProgram

PDA

WWW

Palm Pilot

PocketPC

Printer

editor

Web master

Pdahandler

schedule



Daily events:Daily events:– The web master corrects an error in a web The web master corrects an error in a web

page but the Final Program is not updated.page but the Final Program is not updated.– A room number is changed, but the change A room number is changed, but the change

is not made in the PDA.is not made in the PDA.

So, the third law of KriensSo, the third law of Kriens– Changes that must be made in multiple Changes that must be made in multiple

places are notplaces are not



Coordinating this many publications, Coordinating this many publications, with so many volunteers, is a with so many volunteers, is a nightmare.nightmare.

How can we prevent the need for How can we prevent the need for multiple updates?multiple updates?


The Solution?The Solution?

Committee

OOPSLAOffice

Exhibitions

Participants

CommitteeSub

Committee

Participants

ExhibitorExhibitor

Local info

FinalProgram

AdvanceProgram

PDA

WWW

Palm Pilot

PocketPC

Printer

schedule

?

transformer

transformer

transformer

transformer


mergeXSL

Overall WorkflowOverall WorkflowSource

XML

XSLT

XHTML MIF fileXML2MIF PDFFramemaker

Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Meet XMLMeet XML

What is XML?What is XML?– A standard for describing structured A standard for describing structured

informationinformation

For exampleFor example– Personal informationPersonal information– Congress events and programsCongress events and programs– Book contentBook content


Meet XMLMeet XML

Example Congress Event:Example Congress Event:

– <?xml version="1.0" encoding="utf-8"?><?xml version="1.0" encoding="utf-8"?><event id="onw-1"><event id="onw-1"> <title>Onward!</title> <title>Onward!</title> <authors> <authors> <author> <author> <title>Dick Gabriel</title> <title>Dick Gabriel</title> <email>[email protected]</email> <email>[email protected]</email> </author> </author> <author> <author> <title>Ralph Lauren</title> <title>Ralph Lauren</title> <email>[email protected]</email> <email>[email protected]</email> </author> </author> </authors> </authors> <description>…</description> <description>…</description></event></event>


Meet XMLMeet XML

event

title authors description

author

title email

*


Meet XMLMeet XMLevent

title authors description

author

title email

author

title email

id="onw-1" ...

Dick Gabriel [email protected] Ralph Lauren [email protected]

Onward title!

tag

element

attribute

text


Meet XMLMeet XML

Strong PointsStrong Points– Flexible character encoding, based on Flexible character encoding, based on

Unicode but supporting many other code Unicode but supporting many other code sets:sets: <?xml version="1.0" encoding="iso-8859-1"?><?xml version="1.0" encoding="iso-8859-1"?>

– Based on a text streamBased on a text stream Text files, database, networkText files, database, network

– Defined structure in a Data Type Definition Defined structure in a Data Type Definition (DTD) that simplifies the processing(DTD) that simplifies the processing


Meet XMLMeet XML

Weak pointsWeak points– Very verboseVery verbose– Hard to readHard to read– No clear distinction between attributes No clear distinction between attributes

and sub-elementsand sub-elements <tag><attribute>65536</attribute></tag><tag><attribute>65536</attribute></tag> <tag attribute="65536"/><tag attribute="65536"/>

– A lot of rope to hang yourselfA lot of rope to hang yourself


mergeXSL

The XSL partsThe XSL partsSource

XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

PluckerPalmfiles

HTML webpagesXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Meet XSLTMeet XSLT

XSL is driven by a "stylesheet"XSL is driven by a "stylesheet"– Read 1 or more input filesRead 1 or more input files– Create 1 or more output filesCreate 1 or more output files

What can we do with XSLT?What can we do with XSLT?– Reorder, Add fixed elements, SelectReorder, Add fixed elements, Select– Merge, CalculateMerge, Calculate

This is possible because the data in an This is possible because the data in an XML file has now a name through the tagXML file has now a name through the tag


Meet XSLTMeet XSLT

Structure versus MarkupStructure versus Markup In HTML, tags are used for mark-upIn HTML, tags are used for mark-up

<h1>Onward!</h1><h1>Onward!</h1>

In XML, tags are used for semanticsIn XML, tags are used for semantics <email>[email protected]</email><email>[email protected]</email>

The tag (and its position) identify the The tag (and its position) identify the meaningmeaning of the element. of the element.

Transformers transform the Transformers transform the meaningmeaning into into markupmarkup


Meet XSLTMeet XSLT<?xml version="1.0" encoding="ISO-8859-1"?><conference> <program id="tut"> <title>Tutorials</title> <chair>Sherman Alpert</chair> <event id="tut-1"> <title>Objects revisited</title> </event> </program> <program id="des"> <title>DesignFest</title> <chair>Gail Harris</chair> <event id="des-1"> <title>E-commerce</title> </event> <event id="des-2"> <title>Counting Fries</title> </event> <event id="des-3"> <title>Cellular Networks</title> </event> </program></conference>

Index

DesignFest

1. Cellular Networks

2. Counting Fries

3. E-commerce

Tutorials

1. Objects Revisited


Meet XSLTMeet XSLT

The XML coded data is neutral to the The XML coded data is neutral to the application so different transformers can application so different transformers can markup for:markup for:– Different type of documentsDifferent type of documents

Advance Program, PDA, Final Program, WebAdvance Program, PDA, Final Program, Web

– Different formatsDifferent formats Framemaker, XSL:FO, Word, PDFFramemaker, XSL:FO, Word, PDF

Transformers can be Transformers can be anyany program but program but XSL is designed for this purposeXSL is designed for this purpose


Meet XPathMeet XPath

Selects elements from a documentSelects elements from a document Provides FunctionsProvides Functions ExamplesExamples

– //title//title– //author[email]//author[email]– count(author)>=1count(author)>=1– //program[@id='tec' or @id='pap' or //program[@id='tec' or @id='pap' or

@id='onw' or @id='pra' or @id='pan' or @id='onw' or @id='pra' or @id='pan' or @id='web']/event@id='web']/event


Meet XSLTMeet XSLT

XSLT is an XML file that describes XSLT is an XML file that describes how another XML file must be how another XML file must be transformedtransformed<?xml version="1.0" encoding="ISO-8859-1"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:template match="conference"> <h1>Index</h1> <xsl:for-each select="program"> <xsl:sort select="title"/>

<h2><xsl:value-of select="title"/> (<xsl:value-of select="count(event)"/>)</h2><ol>

<xsl:for-each select="event"> <xsl:sort select="title"/> <li><xsl:value-of select="title"/></li> </xsl:for-each></ol>

</xsl:for-each> </xsl:template></xsl:stylesheet>


Meet XSLTMeet XSLT <?xml version="1.0" encoding="ISO-8859-1"?><?xml version="1.0" encoding="ISO-8859-1"?>

Defines that we use XML version 1 and Defines that we use XML version 1 and use the standard ISO-8859-1 character use the standard ISO-8859-1 character set in the fileset in the file

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">version="1.0">

Defines a stylesheet elementDefines a stylesheet element The xsl: is called a namespaceThe xsl: is called a namespace The URL uniquely identifies the xsl prefixThe URL uniquely identifies the xsl prefix


Meet XSLTMeet XSLT <xsl:template match="conference"><xsl:template match="conference">

A template describes a transformation. This A template describes a transformation. This template will match the conference element template will match the conference element from the inputfrom the input

Inside the xsl:template element, the Inside the xsl:template element, the conference is the conference is the currentcurrent element. element.

<h1>Index</h1><h1>Index</h1>

Insert a fixed text header. Because this Insert a fixed text header. Because this element does not have the xls: namespace, element does not have the xls: namespace, it is inserted in the outputit is inserted in the output


Meet XSLTMeet XSLT <h2><xsl:value-of select="title"/></h2><h2><xsl:value-of select="title"/></h2>

Insert the title of the program. This is a Insert the title of the program. This is a program title.program title.

<ol><ol>

Insert the tag to number the following Insert the tag to number the following elementselements

<xsl:for-each select="event"><xsl:for-each select="event"> <xsl:sort select="title"/><xsl:sort select="title"/>

Sort the events on title and iterate over Sort the events on title and iterate over them. Each event under a program them. Each event under a program becomes the current elementbecomes the current element


Meet XSLTMeet XSLT <xsl:for-each select="program"><xsl:for-each select="program">

<xsl:sort select="title"><xsl:sort select="title">

Repeat for each element under Repeat for each element under conference that has the name conference that has the name "program". The sort element indicates "program". The sort element indicates which field should be sorted.which field should be sorted.

Inside the xsl:for-each element, each Inside the xsl:for-each element, each program is selected and becomes the program is selected and becomes the current element.current element.


Meet XSLTMeet XSLT <li><xsl:value-of select="title"/></li><li><xsl:value-of select="title"/></li>

Output the event titleOutput the event title </ol></ol> </xsl:for-each></xsl:for-each> </xsl:template></xsl:template> </xsl:stylesheet></xsl:stylesheet>

Close all the open elementsClose all the open elements


DemoDemo


mergeXSL


XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Meet MIFMeet MIF

XSLT only restructuresXSLT only restructures No formatting and paginationNo formatting and pagination This is delegated to XSL:FO, which is This is delegated to XSL:FO, which is

still immaturestill immature Framemaker is Framemaker is muchmuch better better

– Large documentsLarge documents– Centralized paragraph and character stylesCentralized paragraph and character styles– etc.etc.


Meet MIFMeet MIF

MIF is a generic import format for MIF is a generic import format for FramemakerFramemaker– Framemaker version independentFramemaker version independent

Gets formatting from templateGets formatting from template Pure text, similar to XML, but not the samePure text, similar to XML, but not the same

<MIFFile 6.00> <Para <PgfTag h1> <ParaLine <String `Conference At A Glance'> > #ParaLine > #Para


Meet MIFMeet MIF

Close to XML Close to XML Uses < and >Uses < and > Very difficult to generate MIF with Very difficult to generate MIF with

XSLT because XSLT because – < and > must be escaped into < and < and > must be escaped into < and

>>– Character encoding issuesCharacter encoding issues– XSLT is a lousy programming language!XSLT is a lousy programming language!


Meet MIFMeet MIF

Framemaker can be used interactively Framemaker can be used interactively or in batch modeor in batch mode– also on PC with dzbatcheralso on PC with dzbatcher

Generates PDFGenerates PDF Provides a very rich formatting Provides a very rich formatting

environmentenvironment StableStable MatureMature


mergeXSL


XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Meet XHTMLMeet XHTML

XHTML is similar to HTMLXHTML is similar to HTML XHTML is valid HTML and XMLXHTML is valid HTML and XML All tags are closedAll tags are closed

– Not <br> but </br>Not <br> but </br>– Not <p>…<p>…<p>… but Not <p>…<p>…<p>… but

<p>…</p><p>…</p><p>…</p><p>…</p>

Has XML headerHas XML header


mergeXSL


XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Meet JavaMeet Java

Not all conversions were feasible in Not all conversions were feasible in XML/XSLTXML/XSLT

This happens when an input or output This happens when an input or output file is not in XML formatfile is not in XML format– XSLT can not handle this wellXSLT can not handle this well

ProgramsPrograms– XML2MIFXML2MIF– DriverDriver


Meet XML2MIFMeet XML2MIF

Developed a Java Program to convert an Developed a Java Program to convert an XML file (with HTML like schema) to MIFXML file (with HTML like schema) to MIF

Tested several third party apps, none Tested several third party apps, none workedworked

Java is ideal for these kind of applicationsJava is ideal for these kind of applications– FreeFree– Unicode supportUnicode support– Many XML parsers availableMany XML parsers available


Meet XML2MIFMeet XML2MIF

Character encodingCharacter encoding Converting text to MIF paragraphsConverting text to MIF paragraphs

– Automatic <Para tag creation (and closing)Automatic <Para tag creation (and closing) Whitespace insertion and removalWhitespace insertion and removal Handling tabs (which is not XHTML)Handling tabs (which is not XHTML) Converting the XHTML tags to MIF tagsConverting the XHTML tags to MIF tags

– Not straightforward due to different markup Not straightforward due to different markup modelsmodels


Meet XML2MIFMeet XML2MIF<p class="time">13:30-17:00, Thursday afternoon</p><p class="entry">Tutorials: How to Use Design Patterns In Java and .NET,<a href="#tut-49"/>

<Para<PgfTag time><ParaLine<String `13:30-17:00, Thursday afternoon'>> #ParaLine> #Para<Para<PgfTag entry><ParaLine<String `Tutorials: How to Use Design Patterns In Java and .NET, '>> #ParaLine<ParaLine<XRef<XRefSrcFile `advance.mif'><XRefName `Page'><XRefSrcText `tut-49'>> #XRef> #ParaLine> #Para

<Para PgfTag="time"> <ParaLine> <String>13:30-17:00, Thursday afternoon</String> </ParaLine> </Para><Para PgfTag="entry"> <ParaLine> <String> Tutorials: How to Use Design Patterns In Java and .NET, </String> </ParaLine> <ParaLine> <XRef XRefSrcText="`tut-49'" XRefName="`Page'" XRefSrcFile="`advance.mif'"/> </ParaLine></Para>


Meet DriverMeet Driver

Bad name!Bad name! The web master had coded previous sites in The web master had coded previous sites in

HTMLHTML This HTML was not easy to re-code in XHTML so This HTML was not easy to re-code in XHTML so

we could not parse itwe could not parse it Driver reads an HTML file and inserts Driver reads an HTML file and inserts

information from the XHTML in appropriate information from the XHTML in appropriate placesplaces– titletitle– bodybody


Meet DriverMeet Driver


mergeXSL


XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Meet PluckerMeet Plucker

A public domain HTML reader for Palm PilotsA public domain HTML reader for Palm Pilots Requires simplified HTMLRequires simplified HTML Consists of a conversion program that reads Consists of a conversion program that reads

an HTML file and compacts all linked files in an HTML file and compacts all linked files in to an archiveto an archive

Converts imagesConverts images Can be downloaded from www.plucker.orgCan be downloaded from www.plucker.org Has some usability problemsHas some usability problems


Meet CVSMeet CVS

CVSBuild

systemWeb

server

programmerscommittee

Publicity chairEditor

MIF files

XML filesXML files and PDFXML files

XML, HTMLand PDF files

XSL files


Meet CVSMeet CVS

Concurrent Version SystemConcurrent Version System Allows multiple people to edit files Allows multiple people to edit files

simultaneouslysimultaneously Revisions are automatically mergedRevisions are automatically merged Conflicts must be manually resolvedConflicts must be manually resolved Triggers a build after every changeTriggers a build after every change

– Web pages, MIF files, PDA filesWeb pages, MIF files, PDA files Very friendly client: TortoiseCVSVery friendly client: TortoiseCVS

– Integrates with MS ExplorerIntegrates with MS Explorer


TortoiseCVSTortoiseCVS


mergeXSL

PreparingPreparingSource

XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


AuthoringAuthoring

Every chair has his own XML fileEvery chair has his own XML file– No editor was providedNo editor was provided– Not everybody can handle text files wellNot everybody can handle text files well

Was simple but powerfulWas simple but powerful


XML Scheme for EventsXML Scheme for Eventstop

program

event

locationtitle

title

author

chair abstractsmallabstract xref

subevent abstractsmallabstract

title abstractsmallabstractauthor

category

track formatlevel

authoror chair

rolenametitle affiliationimg

*

*

*

*

* **

*

* O O

O

O

O O

O

*


AuthoringAuthoring

Validation can be done in Internet Validation can be done in Internet ExplorerExplorer

Next year we need more up-front Next year we need more up-front validation because chairs made validation because chairs made many errorsmany errors



Merge and insertMerge and insert

The XML data files are merged into a The XML data files are merged into a single XML filesingle XML file

During the merge, calculated fields were During the merge, calculated fields were added to simplify later processingadded to simplify later processing

Overall information came from a separate Overall information came from a separate overall.xml fileoverall.xml file– Calendar with day names (Thursday)Calendar with day names (Thursday)– TracksTracks– Included programsIncluded programs


mergeXSL

Printed MaterialPrinted MaterialSource

XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Creating the AP and FPCreating the AP and FP

The all.xml is transformed into a XHTML The all.xml is transformed into a XHTML filefile– Looks like HTMLLooks like HTML– Can be viewed in browserCan be viewed in browser

The result is transformed into an XMIF fileThe result is transformed into an XMIF file– Has the MIF structure with frames but is still Has the MIF structure with frames but is still

XMLXML The XMIF file is transformed into a MIF file The XMIF file is transformed into a MIF file

with the HTML2MIF converterwith the HTML2MIF converter


Creating the AP and FPCreating the AP and FP

The MIF file is automatically made The MIF file is automatically made available on the web when its sources available on the web when its sources have changedhave changed

Editor fetches it and Editor fetches it and – Applies templateApplies template– Does manual editing (!)Does manual editing (!)– Creates PDFCreates PDF– Stores this on the server via CVSStores this on the server via CVS– Result becomes available on the webResult becomes available on the web


AP/FP StructureAP/FP Structure

AP

Intro Glance By TrackTechnicalProgram

Intro Glance Speakers Papers PractionersPanels OnwardWeb

ServicesDetails

TP

TutorialsWorkshopDesignfestEducators

SymposiumDoctoral

SymposiumDemos

DoctoralSymposium

Posters

StudentCompet.

SpecialEvents

StudentVolunteers

Exhibits At a glance

Event details

Summary




mergeXSL

The Web SiteThe Web SiteSource

XML

XSLT


Template

SourceXML

XSLT XMIF

XSLT

XSLT

HTML

HTML Driver

Plucker PRC

HTMLXHTML

HTML

HTMLHTML

XSLT

AllXML

web.xsl

pda.xslap.xslap.xsl

driver.xml

html2mif.xsl

overallXML


Web pagesWeb pages

The web.xsl XSLT script reads the all.xml The web.xsl XSLT script reads the all.xml file and generates a page forfile and generates a page for– Each programEach program– Each eventEach event– At a glance by track (defined in overall.xml)At a glance by track (defined in overall.xml)– By day (also defined in overall.xml)By day (also defined in overall.xml)

All pages are then merged with a All pages are then merged with a templatetemplate



PDAPDA

First generated web pages that were First generated web pages that were optimized for small devicesoptimized for small devices

These were made available on theThese were made available on theweb for AvantGo users.web for AvantGo users.

These pages were converted intoThese pages were converted intoplucker and made availableplucker and made availableas downloads as downloads



Tools UsedTools Used

Saxon XSLT: saxon.sourceforge.netSaxon XSLT: saxon.sourceforge.net Framemaker: www.adobe.comFramemaker: www.adobe.com Java: java.sun.comJava: java.sun.com CVS: CVS: www.cvshome.orgwww.cvshome.org TortoiseCVS: client TortoiseCVS: client

www.tortoisecvs.orgwww.tortoisecvs.org Apache web server: www.apache.orgApache web server: www.apache.org Plucker: www.plucker.orgPlucker: www.plucker.org


ConclusionConclusion

Overall system worked very wellOverall system worked very well The web pages were only partly used due The web pages were only partly used due

to "political" problems, but they were fully to "political" problems, but they were fully generatedgenerated

XML was OK for this crowd, but it is XML was OK for this crowd, but it is probably necessary to use a web page for probably necessary to use a web page for editing with validationediting with validation

Many more possibilitiesMany more possibilities– Audio/Visual, room assignments, signageAudio/Visual, room assignments, signage


QuestionsQuestions

??

Contact:Contact:[email protected]@aQute.

sese+46 300 39800+46 300 39800

Pragmatic XML By Peter Kriens, CEO aQute .

Documents

xml portland

xml file

overall xml slide

pragmatic xml

pda slide

tag slide

markup slide

content slide