Semantics in Declarative System The Evolution of Business Unit Empowerment Dan McCreary Dan McCreary & Associates Wednesday, 5/23/2007 8:00 AM - 9:00 AM Level: Business/Strategic M D Metadata Solutions
Mar 26, 2015
Semantics in Declarative SystemThe Evolution of Business Unit Empowerment
Dan McCrearyDan McCreary & Associates
Wednesday, 5/23/2007 8:00 AM - 9:00 AM
Level: Business/Strategic
M
D
Metadata Solutions
2
M
DMetadata Solutions
Presentation Summary
“Declarative programming” has become the latest buzzword to describe languages that abstractly define systems requirements (the what) and leave the implementation (the how) to be determined by an independent process. This makes the semantics (meaning) of declarative data elements even more critical as these systems are shared between organizations. This presentation:
– Provides a background of declarative programming
– Describes why understanding the semantic aspects of declarative systems is critical to cost-effective software development
Note: All opinions stated in this paper are solely those of the author.
3
M
DMetadata Solutions
Presentation Summary (cont)
– Discusses declarative and semantic aspects of common development systems such as; XHTML, CSS, XForms, XML transforms, XML Schemas, OWL, metadata registries, web services, composition, service-oriented architectures (SOA) and the enterprise service bus (ESB)
– Discusses how social networking software and Wikis are used to quickly build consensus on precise semantics
– Presents ten specific recommendations to lower costs of agile information systems
4
M
DMetadata Solutions
Presentation Includes
• Definitions of declarative systems and contrasts with them with traditional procedural systems and stand-alone declarative languages
• A critical analysis of semantics in declarative systems • Case studies using XForms, Wikis and other
collaborative software• The role of social networking systems, reputation and
trust in the development of semantically precise declarative frameworks
• Specific recommendations of how organizations can be more effective by integrating semantics and declarative systems into their software development processes
5
M
DMetadata Solutions
Evolution Metaphors
• Specialization of Languages
• Generalization of Languages
6
M
DMetadata Solutions
Evolution: Specialization• Darwin’s Galapagos Finches• Beaks are highly adapted to different food sources • Finches adapted to specific ecological "niches“ over
millions of years of isolated evolution
• Similar to domain-specific declarative languages
See Wikipedia "Darwin's Finches"
7
M
DMetadata Solutions
Evolution: Generalization
• Generalization: The Raccoon– The world has a higher population of raccoons today
due to their ability to quickly adapt to changing urban environments
– Similar to highly adaptive procedural languages
8
M
DMetadata Solutions
Computer Science Abstractions
Assembly
1010001010
Fortran
FOR I = 1 TO 10 DO
Structured
Function(A, B)
Object
Class MyClass(Method MyMethod
Declarative<xf:input> <xf:label>
time
Higher abstractions
9
M
DMetadata Solutions
The Software Development Process
• Requirements are about “What”• Design and Build is about “How”
Requirements(BA)
Design(Architect)
Build(Programmer)
Test(QA Staff)
Declarative Languages
10
M
DMetadata Solutions
A Declarative “System” Is…• A software development system, tailored to a specific domain
(such as web applications), used to capture precise business requirements within the context of a problem domain (the implicit context)
• Declarative systems do not specify how requirements are implemented to build working systems. Declarative systems only define the requirements
• Declarative systems document requirements in specialized vocabularies and can be used to generate entire working systems including user interfaces, persistence and test data
• Declarative systems specifically omit some assumed requirements (such as system availability, performance, reliability, security etc)
A Declarative system is a set of "little languages" with precise semantics that fit together like a puzzle to solve a problem
11
M
DMetadata Solutions
Computer Science Definition
• Do not confuse a “Declarative System” with the computer science language taxonomy “Declarative Language”
• “Declarative languages" are used to describe a group of programming languages and to contrast them against imperative languages.
DeclarativeLanguages
FunctionalLanguages
LogicLanguages
ConstraintLanguages
Have sub-types
See Wikipedia “Declarative Programming”
12
M
DMetadata Solutions
Declarative Systems and Context
• Declarative Systems are specialized languages for capturing requirements within a specific domain
• Just as the word “play” connotes meaning based on context* (i.e. theater vs. a playground), a given vocabulary has the ability to capture requirements based on the current problem
• The vocabulary for capturing electronic form requirements (XML Schema) may not be appropriate for expressing your build process (Apache Ant)
* See: http://wordnet.princeton.edu
13
M
DMetadata Solutions
HTML, CSS and SQL
• HTML, CSS, XQuery and SQL are declarative languages using these definitions
• The semantics or “meaning” of each tag in the file is determined by an external organization
• The syntax does not have to be XML
<?xml version="1.0" encoding="UTF-8"?><html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>Declarative Systems</title> </head> <body> <h1>Introduction to Declarative Systems</h1> <p class="author">Dan McCreary</p> <p class="date">April 2007</p> </body></html>
HTML
/* global CSS used by all web pages */body { font-family: Arial, Helvetica, sans-serif; font-size: 75%; margin: 0; padding: 0; width: 1000px;}h1 { color: blue; padding: 0 15px;}
CSS
-- Old expensive booksSELECT title, author, price, pubyearFROM booksWHERE price > 100 AND
pubyear BEFORE ‘1960’ORDER BY title
SQL
xquery version "1.0";<html><body><h1>Old Expensive Books</h1><ul>{for $book in doc("books.xml")//bookorder by $book/titlereturn<li> {$book /title}, {$book}/author}, {$book}/price}, {$book}/pubyear}</li>}</ul></body></html>
XQueryhttp://www.w3.org/TR/REC-html32#body
14
M
DMetadata Solutions
XML Schema Sample
• Screen capture of Altova XML Spy• 30 minutes to learn graphical notation
See: http://www.altova.com/products/xmlspy/graphical_xml_schema_editor.html
15
M
DMetadata Solutions
Use Case: Electronic Forms• User fills out a web-
based form• Typical requirements
may include listing of data to be gathered, data types, validation codes and data repetition patterns
• Examples:– HTML Forms– XForms– InfoPath™
16
M
DMetadata Solutions
Forms Players
<html> <head> <xf:model> <xf:/model> <head> <body> <xf:input> </xf:input> </body></html>
My-XForm.xhtmlRendered
Form
XForms“Players”
XFormsExtension
Mobile Client
Forms Server
Netfront Mobile
See Wikipedia “XForms”
17
M
DMetadata Solutions
What Is Declarativeness for A Context?
• Efficiency at capturing the testable business requirements in a semantically precise and concise manner
• Example: Is a field optional? Will validation of the data fail if the field is missing?
• Ability of each data element to have precise meaning over time and within organizations
Schema Drawing Tool
XML Schema File (.xsd)
XForms
xf|input:required {font-weight: bold;} xf|input:required .xf-value {background-color:#fff6af;}*:required::after {font-weight:bold; font-size:1.5em; content: "*"; color: red; }
XForms CSS tags
18
M
DMetadata Solutions
Declarative Spectrum
• For any given context different “languages” have different levels of “declarativeness”
• General purpose languages are less abstract but can solve a wide variety of problems
• Declarative languages have a more narrow purpose and target a specific problem like styling a web page or selecting data
HTML
JavaAssemblyLanguage C C++
JavaScript
C#
Narrow Purpose
More Abstract
General Purpose
Less Abstract
CSS
XSL XML Schema
Apache Ant
XForms
XQuery
XQuery Update SQL
XPath
Rubyperl
Groovy
Python
XUL
19
M
DMetadata Solutions
Middle-Tiers Remain Procedural
• When the interface is consistent, declarative languages flourish• Middle tiers tend to have the most variation• Wikipedia lists over 200 web application frameworks• CMSMatrix.org lists over 700 content management systems
userdatabase
declarativeness
Presentation/style(client tier)
Data definition, insert,selection and update(persistence tier)
Business Logic(middle tier)
See: http://en.wikipedia.org/wiki/List_of_web_application_frameworks and http://www.cmsmatrix.org
20
M
DMetadata Solutions
The Application Development Puzzle
SemanticsConstraints
Presentation
Build
Publish
Transform
WorkflowQuery
UpdateForms
21
M
DMetadata Solutions
Filling In Each PieceWith A Declarative Language
XMLSchemaCSS
HTML
ApacheAnt
MetadataRegistry
Cocoon
XSL
BPELXQuery
XQuery UpdateXForms
22
M
DMetadata Solutions
Semantically Precise Vocabularies
Standard Domain
Atom/RSS Syndication/Notification
ACORD Insurance Industry Forms Data
FpML Financial Products Markup Language
FixML Banking (Financial Information eXchange)
HL7 Healthcare (now using XML)
MISMO Mortgage Industry
NIEM National Information Exchange Model (registry)
OTA OpenTravel Alliance (Travel industry)
SIF School Interoperability Framework
GJXDM US Department of Justice (registry)
XBRL XML Business Reporting Language
See: http://en.wikipedia.org/wiki/Category:XML-based_standards
23
M
DMetadata Solutions
PhonePhone
Metadata Shopping Tools
AddressAddress
FirstNameFirstName
See http://niem.gtri.gatech.edu/iepd-ssgt/SSGT-SearchSubmit.do
• You don’t need to know about 100,000 SKUs to purchase 10 items from a grocery store
• Sub-schema generation tools give you exactly what you need and nothing more
24
M
DMetadata Solutions
Criteria for Semantic Precision
• Is there a published standard?
• Are there ISO/IEC 11179 definitions?– concise, precise, non-circular, distinct
• Are people using it?– Do a Google search– > 100,000 and you are safe– < 10,000 and you should be concerned
Examples: filetype:owl, filetype:xsd
See “Metadata publishing” Wikipedia
25
M
DMetadata Solutions
If You Use Industry Standards…You Could Be Almost Done…
• If you use industry standards…– and these standards publish their documents in XML
Schema format…– and these standards have been transformed from XML
Schema to XForms…– and you use native XML databases to store and
XQuery to report on the data…• …then sample applications have been created
and do not require additional procedural code– just change the constraints in the XML Schema and
rerun the transforms
See also: http://www.alphaworks.ibm.com/tech/purexml
See: http://www.exist-db.org
26
M
DMetadata Solutions
Accessibility
Business Analysts
SMEs and GUI Tools Users
• Precisely specifybusiness requirements
• Requires data stewardshiptraining
ProceduralProgrammers
• Extend declarativevocabularies andprovide web service“glue”
Lower costs by moving routine logic maintenance to lower levels in the pyramid
ITStrategists
• Architecture andStrategy
(prevent unnecessaryprocedural code)
27
M
DMetadata Solutions
Java Libraries
10,000 class and 100,000+ methods available…but which ones are relevant to your business problem?
28
M
DMetadata Solutions
“Less is More” • XForms 1.1 has
only 21 XML elements
• Much of the presentation of XForms is deferred to CSS
• Event management is deferred to the XML Events
• XML binding is deferred to the XBL standard
See Wikipedia “Minimalism”
Mies van der RoheReconstruction of the
German Pavilion in Barcelona
29
M
DMetadata Solutions
Learning XForms Vocabulary
1. Bind2. Case3. Input4. Instance5. Group6. Help7. Hint8. Label9. Load10.Output
11.Message12.Model13.Repeat14.Secret15.Select16.Select117.Switch18.Submission19.Submit20.Textarea21.Trigger
Source: W3C XForms Quick Reference http://www.w3.org/MarkUp/Forms/2006/xforms-qr.html
30
M
DMetadata Solutions
Recognizing XForms Structures
1. Bind2. Case3. Input4. Instance5. Group6. Help7. Hint8. Label9. Load10.Output
11.Message12.Model13.Repeat14.Secret15.Select16.Select117.Switch18.Submission19.Submit20.Textarea21.Trigger
Color coding limited vocabularies can increase the speed of pattern matching.Look for advanced text editors to provide custom element coloring.
31
M
DMetadata Solutions
The New Semantics of "Nutshell"
• The 1.4 release of Java 2 Standard edition increases the size of the platform by 50%, to 2,757 classes in 135 packages
• 1.5 and 1.6 add additional classes
nutshell: something of small size, amount, or scope in a nutshell : in a very brief statement
992 pages
32
M
DMetadata Solutions
Procedural Programming is Not “Poison”• It would be a mistake to tell all
your procedural programmers that the programs they are creating are fundamentally evil
• The relevant questions are:- How closely does it fit the problem
domain?- Can BAs, SMEs and other non-
programmers maintain the business rules?
- What are the chances that others will be able to maintain it in future years?
- How good are the development tools for your system?
33
M
DMetadata Solutions
Popular Language Have Better Tools
• Editor– Syntax coloring
• Debugger– Set breakpoints– View internal state variables
• Refactoring tools– Can recognize reoccurring
patterns and suggest alternatives
• Performance– Code profiling
demandLimited supply curve
$
quantity
Limited resource cost curve
Whuffie curve
Whuffie is a reputation basedcurrency. Prices drop asdemand increases.
The higher demand for a good debugger, the better open-source products will become.
34
M
DMetadata Solutions
If You Give a Kid a Hammer…
• People solve problems using familiar tools
• People develop specific Cognitive Styles* based on training and experience
• What are we teaching the next generation of developers?
* Source: Shoshana Zuboff: In the Age of the Smart Machine (1988)
…the whole world becomes a nail
35
M
DMetadata Solutions
Use Case: Build Scripts
• Instructions for compiling source code or transforming data
• Vocabulary includes terms such as build, compile, transform, copy or clean
• Examples:– Apache Ant– Apache Maven– UNIX™ make
36
M
DMetadata Solutions
Use Case: Data Selection With XQuery• Ways to specify what data
you want to extract from a data set
• Typical tasks include selecting attributes (columns), filtering, restricting results and changing sort order
• Examples– Structured Query Language
(SQL)– XQuery (w3c standard)– FLOWR
37
M
DMetadata Solutions
Semantics
• The science of meaning
• What you mean when you say “cat”
• How do you associate meaning with symbols (verbal, physical, textual)
• How do we know if we both mean the same thing when we use a word?
• What if a word has multiple meanings?
Reference: WordNet
38
M
DMetadata Solutions
Semantic Triangle
• Symbols can only link to referents through concepts• You can not link directly from a symbol to a referent
referent
concept
symbol
“cat”
Wikipedia: Semiotic triangle
39
M
DMetadata Solutions
Communication
• Communication involves exchanging symbols that describe common attributes
• A one-to-one match of attributes that describe a common symbol match implies a high precision match
“cat”
• Domestic feline• House pet• Has fur• Has whiskers• Sometimes has fleas• Chases mice
• Domestic feline• House pet• Has fur• Has whiskers• Sometimes has fleas• Chases mice
40
M
DMetadata Solutions
Same symbol – different meaning
• Little match of common attributes• Low precision semantic match
“customer”
• First name• Last name• Home address• E-mail• Cell phone number• Gender
• Company name• Home office address• Branch office address• CEO name• Web site• Industry code
41
M
DMetadata Solutions
High and Low Precision
• The ideal is to have a precise standard and to use the data elements exactly as they were intended
vague standardvague usage
Not-so-good
precise standardvague usage
better
precise standard precise usage
best
42
M
DMetadata Solutions
Semantic Precision• Semantic mappings are relative in time and
between groups of people (organization)• Semantic variability over time
– Something that has precise meaning to you today may not have the same precise meaning a year from now
– Our memories are imperfect and change over time• Semantic variability across organizations
(project and organization)– A “customer” to one organization may denote a
person but to another organization it may denote a company
43
M
DMetadata Solutions
Semantics of an XML Data Element
• A developer puts an XML data element in an xml file• The tag has some meaning and the data within the tag
has some meaning when it was created by the developer• What is the probability
– That the developer will know the same meaning of the code 47 one year later
– That another project that opens file will understand and be able to use the meaning of the tag
– Vague standards often trigger vague interpretations of the meaning of data
<code>47</code>
44
M
DMetadata Solutions
Semantic Precision in Space and Time
time
space: (projects, organizations)
Small SemanticFootprint
(rapid prototype)
LargeSemanticFootprint
(long lifetimesystems)
weeks months years 10+ years
person
team
dept.
world
enter-prise
45
M
DMetadata Solutions
DRY Coding, XSL and MDA• If developers can’t quickly transform it…
…they will copy it.• DRY: Don’t Repeat Yourself• Documentation
– Is always kept up to date?– Do developers communicate their intent?– JavaDoc – generated from the source code
• The tendency to copy and paste is just too common– Developers must be diligent– Budgets must be adequate– Time must be sufficient
• You can promote Model-Driven Architecture (MDA) by reducing the effort of transformation from specifications captured in declarative languages
46
M
DMetadata Solutions
Use Procedural “Glue”
• XForms uses a REST interface to send XML to a server• Some XML databases (DB2 v9) sill use JDBC interfaces to insert
XML documents• Use procedural glue to build custom interfaces between systems
with incompatible interfaces
Procedural “Glue”
XQuery Update(JDBC)
XForms XML(REST)
47
M
DMetadata Solutions
How Quickly Can We Create New Declarative Languages?
• Configuration files are really just small languages
• Configuration files are easy to parse, validate data using XML tools and build custom forms to use
• Easy to teach non-programmers to graphically build XML Schemas to validate XML files– Answer – About a week
48
M
DMetadata Solutions
How Quickly Can We Create Consensus?
• Example: XForms standard– Work started in 2001– XForms 2.0 is still a two to three years away
• How can we accelerate this consensus building process?
What factors impact the rate that species evolve?What impact does life span have on a species?How are design ideas exchanged between species?
49
M
DMetadata Solutions
Solution: Wikis and Collaboration
• How long does it take to build consensus on the semantics of a new data element?
• How many people might use this declarative language?
• The larger the stakeholder group, the longer it takes
50
M
DMetadata Solutions
Parker Projection
Time
100%
RelativeCode Base
Procedural code (Java, JavaScript, VB, C#, C++)
Declarative code (XHTML, CSS, XSLT, XForms)
Source: Jason Parker, Minnesota Department of Revenue, November 2006
51
M
DMetadata Solutions
Top 10 Recommendations1. Use semantically-precise declarative systems over procedural systems
when the business problem is in the right context2. Aggressively promote data stewardships within business units3. Use controlled vocabularies and metadata shopping tools to allow non-
programmers to select data elements from lists4. Separate semantics (leaf elements) from constraints (branches)5. Leverage the pattern matching portion of the brain using graphical
development tools and element-based syntax coloring editors6. Train programmers on ways to extend declarative vocabularies using
technologies such as XML Binding Language (XBL)7. Use procedural code to build interfaces between incompatible systems8. Isolate procedural by providing XML/REST service wrappers code behind
semantically precise web services9. Don’t expect your procedural programmers to jump on the declarative
bandwagon without some training10. Use social networking tools (voting, wikis) to rank and find appropriate
tags and extensions
52
M
DMetadata Solutions
Unanswered Questions1. How will declarative systems evolve? Will there be a few or many?2. How quickly will declarative languages evolve? Will they continue to
specialize?3. Can we deliver entire working systems without ever leaving a single
family of standards? (see XForms/eXist and w3c example)4. What will be the impact of semantic wikis and micro-formats?5. Can individual tags stand alone or do they need to be integrated into
larger standards to be accepted?6. Will metadata shopping tools and graphical constraint tools make the
process of building entire systems accessible to mere mortals?7. Will declarative systems halt overseas out-sourcing of procedural
software developers?8. Will there be a large increase in the demand for staff that can extend
declarative systems?9. Will declarative systems allow applications to run anywhere a “Forms
Players” can run?10.Will vendors used to locking customers into proprietary APIs be
forced to compete by having more robust and more reliable execution platforms?
53
M
DMetadata Solutions
Selected References• Evolution
– Charles Darwin, Origin of the Species– Richard Dawkins, The Blind Watchmaker
• Cognitive Styles– Shoshana Zuboff: In the Age of the Smart Machine (1988)
• Building Trust– Succeeding with Objects: Decision Frameworks for Project Management, Adele
Goldberg and Kenneth S. Rubin• XML Data Standards
– NIEM Concept of Operations at NIEM.gov– ISO/IEC 11179 (wikipedia)
• XForms– XForms.org, wikipedia
• Domain Specific Languages– Niel Ford, Language Oriented Programming– Martin Flowler
http://www.infoq.com/presentations/domain-specific-languages