ABAP-XML Mapping

ABAP XML MappingPeter McNulty, NetWeaver Product ManagementSAP Labs, LLCIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummaryIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummaryIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummary SAP AG 2004, SAP TechEd / ABAP 252 / 5From HTML to XMLThe Web is evolving from HTML afixed language for human viewing to XML a generic language for machine processing People navigate through information in the form ofHTML documents.HTMLHTMLHTMLXMLXMLXML Machines exchange data and metadata in the form of XMLdocuments. SAP AG 2004, SAP TechEd / ABAP 252 / 6Example: Slashdot.orgRSS feed (news syndication)URI in:http://slashdot.org/it.rssXML out:as HTMLSet up system reports as RSS feeds more effective viewing with RSS aggregators SAP AG 2004, SAP TechEd / ABAP 252 / 7Example: Xignite Financial Web ServicesRealtime currency exchange ratesURI in:http://www.xignite.com/xCurrencies.asmx/GetRealTimeCrossRateTable?Symbols=USD%2CEUR%2CJPY%2CGBP&InvokeButton=Invoke&xMethod=GETXML out:Call a business service from ABAP Or offer one. SAP AG 2004, SAP TechEd / ABAP 252 / 8Example: Amazon Web ServicesTransactions on the Amazon marketplace URI in:http://xml.amazon.com/onca/xml3?locale=us&t=te&dev-t=te&KeywordSearch=ABAP&mode=books&sort=+daterank&offer=All&type=lite&page=1&f=xmlXML out:Retrieval: GETChange: POST SAP AG 2004, SAP TechEd / ABAP 252 / 9XML from a Programmers Perspective (1)From a programmers perspective, XML is ... a generic data model (XML Infoset) for trees with ordered, structured nodes (elements) unordered, unstructured nodes (attributes) a generic syntax (markup) for representation of trees the basis for an open set of standards and toolsparsing: text tree;rendering: tree texttyping, validating: XSchema, RELAX NGquerying, transforming: XPath, XSLT a generic data structure concept for programming languages primitive: DOM, SAX data binding: JAXB SAP AG 2004, SAP TechEd / ABAP 252 / 10XML from a Programmers Perspective (2)Reasons for using XML handle document-like data (e.g. forms) represent data under heavy schema evolution extensibility, flexibility, persistence combine heterogeneous data "framework" applications, middleware, ... platform-independent modeling "repositories", e.g. SAP WebDynpro ignorance / abuse too lazy to design & implement adequate object model... open communication format SAP AG 2004, SAP TechEd / ABAP 252 / 11XML from a Programmers Perspective (3)Using XML Problems solved standard syntax standard tools can model almost anything Problems gained performance drain no semantics unclear relationship with data structures (data conversion)Part II (XML from/to ABAP) schema inflation4 "Business ML" by standards body X4 "Business ML" by vendor Y4 Proprietary format by application Zneed fortransformations SAP AG 2004, SAP TechEd / ABAP 252 / 12Transforming XML with XSLT (1)Tree transformations with XSLT(a)XML treetoXML tree(data-centric)(b)XML treetotext(document-centric)XML Tree.....................HTML / Text(b)XML Tree(a)XSLT SAP AG 2004, SAP TechEd / ABAP 252 / 13Transforming XML with XSLT (2)XSLT* is ... a high-level tree-transformation language with XML syntax declarative no "state" rule-based pattern matching against source tree functional source tree navigation by XPath expressions compositional result tree construction by XSLT instructions mixed withliteral XML fragments "the SQL of the Web"* "Extensible Stylesheet Language / Transformations" SAP AG 2004, SAP TechEd / ABAP 252 / 14Transforming XML with XSLT (3)bookeditorlast firstnamechaptersectionauthorsectionchapterSmith Johnbook bookhead = The Pain of DOM ProgrammingXPath example: tree navigationbook[editor[name/last='Smith' and name/first='John']]//section[contains(@head,'DOM')]/ancestor::chapter[1]/authorIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummary SAP AG 2004, SAP TechEd / ABAP 252 / 16XML in ABAP: iXML Package (1)The iXML package (since 4.6D): features implemented in kernel (C++) encapsulated in ABAP proxy classes general XML parser / renderer event-based parser (~ SAX) DOM (Document Object Model) superset of DOM level 1 (incl. XML namespaces) Validation DTD (since 6.10)not XMLSchema SAP AG 2004, SAP TechEd / ABAP 252 / 17XML in ABAP: iXML Package (2)iXML interfaces in ABAP Objects OO design: interfaces, inheritance, factories main factory class CL_IXML ~40 InterfacesIF_IXML_PARSER,IF_IXML_NODE, ... documentation:SAP Library [ SAP NetWeaver Components ] SAP Web Application Server Basis Services / Communication XML Library used by many SAP applications SAP AG 2004, SAP TechEd / ABAP 252 / 18XML in ABAP: iXML Package (3)iXML example programs in packageSIXML_TEST parsing into a DOM: T_PARSING_DOM manipulating a DOM: T_DOM_MANIPULATE rendering a DOM: T_RENDERING_DOM validating with a DTD: T_DTD_VALIDATIONdata element type ref to if_ixml_element.element = document->create_element( name = 'date' ).element->set_attribute(name = 'format' value = 'yyyymmdd' ).element->set_value( value = ldate ). SAP AG 2004, SAP TechEd / ABAP 252 / 19XML in ABAP: XSLT Processor (1)The SAP XSLT processor (since 6.10): features performance implementation in SAP kernel (C++) scalability optimization for server-side execution interoperability with iXML package conformance (except for justified omissions) integration into language environmentcall from ABAP, call back to ABAP integration into development environmentworkbench, transport SAP AG 2004, SAP TechEd / ABAP 252 / 20XML in ABAP: XSLT Processor (2)Unimplemented XSLT 1.0 featuresE xsl:numberE Forwards-compatible processing, fallback processingE Attribute setsE Namespace aliasingImplemented XSLT 2.0 featuresE Grouping (xsl:for-each-group)E User-defined XPath functions (xsl:function)E Multiple results (xsl:result-document)Extensions:namespacehttp://www.sap.com/sapxslE ABAP Calls (by extension function or extension instruction)E many extension functionsE XPath 2.0 features SAP AG 2004, SAP TechEd / ABAP 252 / 21XML in ABAP: XSLT Processor (3)Extension example: ABAP Call by extension function from XPath

At top level: Declare external functionIn XPath expression: Invoke instance method on external object

function parameter declarationsfunction result declarationXSLT variable bound to ABAP object XPath parameter values SAP AG 2004, SAP TechEd / ABAP 252 / 22XML in ABAP: Invocation of XSLTABAP statement CALL TRANSFORMATIONTRY.CALL TRANSFORMATIONmy_transor (my_trans_name)PARAMETERS p_1 = my_par_1 ... p_n = my_par_nor (my_par_table)SOURCE XML my_xml_sourceRESULT XML my_xml_result.CATCH cx_transformation_error INTO exc....ENDTRY.static or dynamic call of XSLT programprogram parametersXML source:(x)stringor tableor REF TO IF_IXML_ISTREAMor REF TO IF_IXML_NODEXML result:(x)stringor tableor REF TO IF_IXML_OSTREAMor REF TO IF_IXML_DOCUMENTruntime errors throw exceptions SAP AG 2004, SAP TechEd / ABAP 252 / 23XML in ABAP: XSLT DevelopmentWorkbench integration SE80: Edit object More... Transformation object tree, context menu:Create More... Transformation direct: transaction STRANS (6.20: XSLT_TOOL) testing: transaction XSLT programs must be activated before use check / activate triggers compilation maintenance API: function XSLT_MAINTENANCE SAP AG 2004, SAP TechEd / ABAP 252 / 24DemoDemo Transformations in the ABAP WorkbenchCheck Activate Test Flights example (SSTDEMO2): XML to XML with XSLT XML to HTML with XSLTIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummary SAP AG 2004, SAP TechEd / ABAP 252 / 26XML + ABAP: Application AreasWho needs (de)serialization?XML treedata treeserialization deserialization XML-based communication middleware SAP Exchange Infrastructure (XI) Web Services XML-based persistence SAP Archiving XML repositories DB tablesimport / export SAP WebDynpro metadata application-specific uses XML via HTTP / in database / ... SAP AG 2004, SAP TechEd / ABAP 252 / 27XML + ABAP: Structure MappingThe structure mapping problemoutsideinsideABAP functionalityABAP applicationsexternalapplicationsABAP data structureexternal XML formatXML-based communicationstructure difference SAP AG 2004, SAP TechEd / ABAP 252 / 28XML + ABAP: ApproachesWhich side is driving? inside-out approach outside-in approach symmetric approach (6.20) canonical XML encoding of data structures+ transformation with XSLT symmetric approach (6.40) dedicated XML / data transformation language no conceptual "canonical encoding" indirection SAP AG 2004, SAP TechEd / ABAP 252 / 29XML + ABAP: Inside-outThe inside-out approachABAP functionalityABAP data structure1outsideinsideexternal XML format3external mapping (XSLT...)canonical XML encoding2XML handler6.20 kernel SAP AG 2004, SAP TechEd / ABAP 252 / 30generated[ generated ]6.20 kernel XML + ABAP: Outside-inThe outside-in approach ("data binding")ABAP functionalityABAP data structure1boutsideinsideexternal XML format1aABAP adapter3XML handlerABAP proxyABAP proxy interface2 SAP AG 2004, SAP TechEd / ABAP 252 / 31XML + ABAP: SymmetricThe symmetric approach (6.20)outsideinsideABAP functionalityABAP data structure1external XML format12MappingEngine(6.20 Kernel)XSLT programsXSLT SAP AG 2004, SAP TechEd / ABAP 252 / 32ABAP XML with XSLTOutbound XMLtransform DOM to outbound XMLXSLT(outbound)construct canonical DOM for data treeXMLABAP SAP AG 2004, SAP TechEd / ABAP 252 / 33XML ABAP with XSLTInbound XMLtransform DOM todata tree (directly)XMLconceptual:transformto canonicalXML treeXSLT(inbound)ABAPparse inbound XML to DOM SAP AG 2004, SAP TechEd / ABAP 252 / 34XML + ABAP with XSLT: ScopeScope:HTTP... not a communication mechanism only a mapping engine...WebServices not an RPC frameworkIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummary SAP AG 2004, SAP TechEd / ABAP 252 / 36XML + ABAP: Invocation of XSLTSerializationCALL TRANSFORMATION ...PARAMETERS...OPTIONS option_1 = string ...option_n = stringSOURCEXP_1 = my_var_1 ... XP_n = my_var_nor (my_var_table)RESULT XML my_xml_result.CALL TRANSFORMATION ...PARAMETERS...OPTIONS option_1 = string ...option_n = stringSOURCE XML my_xml_sourceRESULTXP_1 = my_var_1 ... XP_n = my_var_nor (my_var_table).Deserializationbinding ofprogram variablesto symbolic XML names SAP AG 2004, SAP TechEd / ABAP 252 / 37ABAP XML: Invocation of XSLT (Example)ABAP fragmentDATA num TYPE P LENGTH 5 DECIMALS 2.DATA txt TYPE STRING.num = '1.23-'.txt = 'Yes, 2 < 3'. CALL TRANSFORMATION idSOURCE Foo = numbar = txtRESULT XML my_xml_result.XML result fragment...-1.23Yes, 2 < 3...symbolic names are normalized to uppercaseformat depends on corresponding ABAP variableidentity transformation SAP AG 2004, SAP TechEd / ABAP 252 / 38XML ABAP: Invocation of XSLT (Example)ABAP fragmentDATA num TYPE P LENGTH 5 DECIMALS 2.DATA txt TYPE STRING.CALL TRANSFORMATION idSOURCE XML my_xml_sourceRESULT Foo = numbar = txt.WRITE: / num, / txt.XML source fragment...E-1.23EYes, 2 < 3...insignificant whitespaceABAP type determines expected XML content1.23-Yes, 2 < 3 SAP AG 2004, SAP TechEd / ABAP 252 / 39XML + ABAP with XSLT: asXML FormatABAP Types and the asXML Format ABAP asXML :type of ABAP source variable(s)determines generated asXML representation asXML ABAP : type of ABAP result variable(s)determines required asXML representation source / result ABAP variables must be typedA Data Exchange Format XML-Schema datatypes "Human-readability" not a top priority no dependence on customization no dependence on locale settings no use of conversion exits SAP AG 2004, SAP TechEd / ABAP 252 / 40asXML Format: Simple TypesSimple TypesABCDEF456789AB20150120010820-3.14E-21.23-123-001234EHelloEHiEEABAP Example*-1.23decimal Pstring of digits001234string ([0-9]+) Nq83vbase64Binary Xbase64-encoded binary dataRWeJqw==base64Binary XSTRING20:15:01time TISO 8601 date/time2001-08-20date D-3.14E-2double Fnumber-123intunsignedByte, shortIINT1, INT2EHellostring Cstring of charactersEHiEEstring STRINGCommentXML Example XML Schema TypeABAP Type* Internal string representation of ABAP datatypes SAP AG 2004, SAP TechEd / ABAP 252 / 41asXML Format: StructuresStructuresDATA miles TYPE person.miles-name = 'Miles Davis'.miles-born = '19260526'.CALL TRANSFORMATION idSOURCE LEGEND = milesRESULT XML my_xml_result....

Miles Davis1926-05-26

...DDict StructurePERSON:NAME TYPE STRINGBORN TYPE Dcomponent names determine XML sub-element names SAP AG 2004, SAP TechEd / ABAP 252 / 42asXML Format: Internal TablesInternal TablesDATA: one TYPE person.DATA: itab TYPE STANDARD TABLE OF person.one-name = 'John Coltrane'.one-born = '19260923'. APPEND one TO itab.one-name = 'Miles Davis'.one-born = '19260526'. APPEND one TO itab.one-name = 'Charlie Parker'. one-born = '19200829'. APPEND one TO itab.CALL TRANSFORMATION idSOURCE GREATEST = itabRESULT XML my_xml_result....

John Coltrane 1926-09-23Miles Davis 1926-05-26Charlie Parker1920-08-29

...line type from DDict (or ) SAP AG 2004, SAP TechEd / ABAP 252 / 43asXML Format: EnvelopeasXML Envelope

... ... ...

list of ABAP-XML bindings SAP AG 2004, SAP TechEd / ABAP 252 / 44XML + ABAP with XSLT: AdvantagesAdvantages of the XSLT solution no restrictions arbitrary XML schemas arbitrary data & object types4 graphs of objects arbitrarily complex structural transformations no redundancy no generation of schemas from types (schema inflation) no generation of types from schemas (type inflation) each side retains its structures high abstraction level no low-level XML handling in applications separate, expressive transformation language SAP AG 2004, SAP TechEd / ABAP 252 / 45XML + ABAP with XSLT: DisadvantagesDisadvantages of the XSLT solution learning XSLT overkill for simple conversion tasks no tool support asymmetric programs one for XML ABAP, one for ABAP XML resource consumption (Time & Space) T / S : DOM construction (on source side) T : codepage conversions (internal encoding SAP CP) T : XSLT engine overhead (complex state, powerful operations) no static type checkingIntroXML in Open SystemsXML in ABAPABAP XML Mapping: ConceptsABAP XML Mapping with XSLTSimple TransformationsSummary SAP AG 2004, SAP TechEd / ABAP 252 / 47XML + ABAP without XSLT: RequirementsRequirements for a dedicated XML/ABAP mapping engine time: increase throughput by factor 10 space: increase / eliminate upper limit on size of data ease of use simple syntax & semantics, statically type-checked tool support for creating mappings one program for XML ABAP and ABAP XMLDeliberate trade-offs lower expressive power (but cover 90% of typical applications) not usable for XML XML "Simple Transformations" SAP AG 2004, SAP TechEd / ABAP 252 / 48random accesslinear(stream reader)linear(modulo component order)random access (XPath)ABAP resulttree constructionXML sourcetree navigationlinearrandom accesslinearrandom access (XPath on canonical DOM)XML resulttree constructionABAP sourcetree navigationSimple Transformations: Tree AccessTree access in XSLT vs. Simple TransformationsSimple TransfomationsXSLTDeserializationSerialization SAP AG 2004, SAP TechEd / ABAP 252 / 49Simple Transformations: Expressive PowerAnything that can be done with ... accessing each node in the data treeany number of times accessing each node in the XML treeat most once,in document order (with "lookahead 1" on XML source)... which includes (any combination of) ...renamings (e.g.: structure-component / element names)projections (omission of sub-trees)permutations (changes in sub-tree order)constants (e.g.: constant values, insertion of tree levels)defaults (for initial / special values)conditionals (e.g.: existence of sub-trees, value of nodes)value maps covers most data mappings in practice SAP AG 2004, SAP TechEd / ABAP 252 / 50Simple Transformations: Key Features (1)Programs are XML templates literal XML withinterspersed instructions declarative, straightforward semanticsData tree access bynode references instructions access data by simple reference expressions all named children of a data node are accessible by name tables are accessible as a whole (all lines or none)

SAP AG 2004, SAP TechEd / ABAP 252 / 51Simple Transformations: Key Features (2)Programs are reversible serialization : write (produce) tokens to stream deserialization : match (consume) tokens from stream invocation determines direction if no asymmetric construct is used: D[program] ( S[program] (data) )=data S[program] ( D[program] (document) ) document(D: deserialization, S: serialization) SAP AG 2004, SAP TechEd / ABAP 252 / 52From 6.20 to 6.40XSLT is generalized to:Transformation = XSLT or Simple Transformation workbench integration same workbench / transport object type same access paths transformation type determined in creation dialog XSLT_MAINTENANCE works for both types ABAP integration CALL TRANSFORMATION works for both types both types take same XML input / output types(e.g. iXML streams) exception hierarchy generalized (new common root) uniform appearance (easy to switch from XSLT solutions) SAP AG 2004, SAP TechEd / ABAP 252 / 53Simple Transformations: ImplementationVirtual machine (VM) compilation of programs to bytecode storage of bytecode in database / buffering in shared memory interpretation of bytecode in kernel (lean engine) scalable performanceXML stream reader / writer reader: efficient "token pull" discipline specialized for standard encoding UTF-8 also usable directly from ABAP:classes CL_FX_READER, CL_FX_WRITER limit on data size lifted SAP AG 2004, SAP TechEd / ABAP 252 / 54DemoDemoFlights example (SSTDEMO2): ABAP to XML with ST XML to ABAP with ST SAP AG 2004, SAP TechEd / ABAP 252 / 55Simple Transformations: Programs

( )*( )*( variable / parameter declaration )*( [ context declaration ]template content )*

formal root name(s)transformation template(s) SAP AG 2004, SAP TechEd / ABAP 252 / 56ST Constructs: ValueValueS ref-node value to XMLD XML value to ref-node

examplecopy value from/to field CHAR,with special mappings: S map { , +, } to~D map ~to

SAP AG 2004, SAP TechEd / ABAP 252 / 57ST Constructs: Literal Element / AttributeLiteral element / attributeS write literal contentD match literal content(no match error)

[template content]

examplealternative

12:59:00 XML SAP AG 2004, SAP TechEd / ABAP 252 / 58ST Constructs: Literal TextLiteral textS write literal textD match literal texttextexamplealternative

CET

text CET12:59:00 CET XML SAP AG 2004, SAP TechEd / ABAP 252 / 59ST Constructs: AttributeAttributeS write attributeD match attribute

[template content]

example

12:59:00 XML SAP AG 2004, SAP TechEd / ABAP 252 / 60ST Constructs: Deep CopyDeep CopyS copy sub-tree belowref-node to streamD copy sub-tree belowcurrent XML node to ref-node

example

CET12:59:00

EST06:59:00

XML SAP AG 2004, SAP TechEd / ABAP 252 / 61ST Constructs: Table LoopTable LoopS evaluate content for all lines of tableD evaluate content, insert lines into table until matching fails

template content

example

CET12:59:00EST06:59:00XML SAP AG 2004, SAP TechEd / ABAP 252 / 62ST Constructs: Basic Conditional (1)Basic ConditionalS [if data condition is true:]evaluate template contentD [if template content matches:]evaluate template content[establish assertions][evaluate check-clause]

template content

exampleS if TZONE='CET' andTIME = t > 000000:write tD if t:TIME:=t, TZONE:='CET', check TIME > 000000

SAP AG 2004, SAP TechEd / ABAP 252 / 63ST Constructs: Basic Conditional (2)Value Assertions C('string') or'string' D('yyyymmdd') F(float) I(integer) orinteger N('digits') P('decimal') T('hhmmss') X('xstring') initial(ref) ref = constant

ref : general:ref('name') simplified: namefor name of the formletter ( letter | digit | _ )*exampleCOUNT = 1,ref('PRICE.%OFF') = P('7.5')ref-nodes in expressionsconstants SAP AG 2004, SAP TechEd / ABAP 252 / 64ST Constructs: Basic Conditional (3)Check-clauseexpr : ref op ( constant | ref )withop in {=, !=, , =} initial(ref),not-initial(ref) exist(ref) (node-existence check) t-op(ref)with t-op = type-C|D|F|I|N|P|T|X (node-type check) expr and expr expr or expr not(expr) ( expr )

exampleTIME>CTIME and (initial(TZONE) or TZONE='CET') SAP AG 2004, SAP TechEd / ABAP 252 / 65ST Constructs: Composite ConditionalSwitchS evaluate first case with true data conditionD evaluate first case with matching pattern

case*

case*

GroupS evaluate all cases with true data conditionD evaluate all cases with matching patterncase : (only serialization) (only deserialization) SAP AG 2004, SAP TechEd / ABAP 252 / 66ST Constructs: Modularization (1)Variable / Parameter Declarationdeclaration :

declaration*

declaration*

template content

Context of main template: top-level declarations Default context of sub-template: one unnamed root initial / default value in val parameter direction (kind) independent of transformation direction SAP AG 2004, SAP TechEd / ABAP 252 / 67ST Constructs: Modularization (2)Apply Template

binding*

binding :

binding*

Call Transformation tt:apply: named template in main or included program tt:call: existing ST program (separate load) SAP AG 2004, SAP TechEd / ABAP 252 / 68ST Constructs: Variable [De]SerializationRead VariableS no effectD XML value to variable

Write VariableS variable value to XMLD no effect

SAP AG 2004, SAP TechEd / ABAP 252 / 69ST Constructs: AssignmentAssign to Ref-NodeS no effectD assign to ref-node

Assign to VariableS assign to variableD except if ref (no effect)

examplesame effect as

SD

SAP AG 2004, SAP TechEd / ABAP 252 / 70ST Constructs: Variable ConditionalBasic Variable Condition