The Cernunnos Project Andrew Wills JA-SIG 2007 Summer Conference, Denver Monday June 25th, 2007 © Copyright Unicon, Inc., 2006. This work is the intellectual.

The Cernunnos Project

Andrew WillsJA-SIG 2007 Summer Conference, Denver

Monday June 25th, 2007

© Copyright Unicon, Inc., 2006. This work is the intellectual property of Unicon, Inc. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of Unicon, Inc. To disseminate otherwise or to republish requires written permission from Unicon, Inc.

1. Introduction

2. Motivation, Goals, & Strategy

3. Using Cernunnos

4. Cernunnos & uPortal

Introducing Cernunnos

• Cernunnos is an open source Java technology for

implementing, instrumenting, and combining valuable

software behavior

• Project Home Page (Google Code): http://

code.google.com/p/cernunnos/

• Community (Google Groups):

http://groups.google.com/group/cernunnos-discussion/

• Presently, there is one contributor (guess who?)

The Cernunnos Platform

• Cernunnos provides a medley of components out-of-the-box (just like Java):

– A contract for Tasks & Phrases (Java API)

– A basic container implementation for running Tasks & Phrases

– An assortment of useful Task & Phrase implementations

– A core syntax for instrumenting these implementations (XML-based)

– Useful tools (e.g. Grammardoc, Command Line Scripts)

But Isn’t That Just Like …

• Cernunnos has some similarities with popular

technologies like Ant, Jelly, Groovy, etc.

• But there are meaningful differences in the details,

some of which are discussed here:

http://code.google.com/p/cernunnos/wiki/Comparison_with_Jelly

• The creator of Cernunnos sees the project not as a

tool written in Java, but as an innovative way to

practice Java

• From that perspective, it appears more similar to

Servlets or Spring than the items above

Motivation, Goals, & Strategy

Notable Quote

“Java’s not worth building in. Nobody uses

Java anymore. It’s this big heavyweight ball

and chain.”

- Steve Jobs

Taken from a recent New York Times interview

about Apple’s new iPhone

http://www.theserverside.com/blogs/thread.tss?thread_id=45781

Goals of Cernunnos

• Cernunnos was created to address software

professional effectiveness

• Professionals who use Cernunnos should be

able to deliver a higher ratio of software value

(outputs) to effort (inputs) compared with

“industry normal” practices

Goals of Cernunnos (Cont.)

Cenunnos is not about:

• Security

• Scalability

• Usability

Cernunnos is about:

• Getting more business

value from your

investment in custom

software

Strategy in a Nutshell

• Mature disciplines are good at creating high-

level solutions from standardized, low-level

components

• But in software we reinvent low-level

behaviors constantly: JDBC calls, file system

I/O, XML parsing, XSL Transformations, etc.

• Cernunnos provides a standard form factor

for behavior, allowing behaviors (old & new)

to be combined in innovative, flexible ways

Short Backstory

Your client is moving to PeopleSoft

Your Assignment:

1. An existing work study app must be changed

to pull data from a new database

2. The work must be done in Java

3. You must do the work

4. The source database isn’t ready, and won’t

be ready before your contract is up

Using Cernunnos

Tasks & Phrases

• In Cernunnos, meaningful work is handled by

Tasks & Phrases

• A Task is a unit of behavior

• A Phrase is an expression that calculates a

value at runtime

<copy-file location="http://www.google.com" to-file="google_home.html"/>

<echo-ln>${valueOf(.)}</echo-ln>

Tasks & Phrases (Cont.)

<!--Examine the contents of all JAR files in the present directory and all subdirectories. Print the name(s) of all entries that match the specifiedpattern.-->

<file-iterator includes="**/*.jar"> <echo-ln>${req(Attributes.LOCATION)}:</echo-ln> <archive-iterator> <if test="${jexl(Attributes.LOCATION.contains('${req($1)}'))}"> <echo-ln prefix="&#009;">${req(Attributes.LOCATION)}</echo-ln> </if> </archive-iterator> <echo-ln/></file-iterator>

> crn find-in-jar.crn DocumentFactory

Tasks & Phrases (Cont.)

• Call standard (out-of-the-box) Tasks & Phrases by

names from a standard grammar file

• Use the <add-grammar> Task to register a custom

grammar file; or

• Refer to Task & Phrase implementations directly (viz.

fully-qualified class name)

<parameter value="${org.jasig.portal.groups.GetMemberKeyPhrase(${singleNode(child/*)})}“/>

Cernunnos Grammardoc

Installing Cernunnos

Running Cernunnos from the command line:

1. Install the following tools

– Subversion

– Ant

2. Download Cernunnos source

> svn checkout http://cernunnos.googlecode.com/svn/trunk/

3. Compile Cernunnos

> ant build

4. Add the Cernunnos bin/ directory to your path

Embedding Cernunnos

Using Cernunnos in other applications:

1. Add the following JARs to the classpath:

• cernunnos.jar

• commons-jexl-1.1.jar

• dom4j.jar

• jaxen.jar

2. Use the ScriptRunner classScriptRunner runner = new ScriptRunner();

runner.run(location)

Invoking Cernunnos

• You can specify a Cernunnos script using

either a file system path, a URL, or a location

in the classpath

• This is not only true for scripts – you can

reference pretty much any resource pretty

much anywhere in any of these ways

> crn grammardoc.crn

> crn http://cernunnos-discussion.googlegroups.com/web/find-class-in-jar.crn Document

> crn classpath://org.jasig.portal.container.deploy.deploy-portlet-app.crn myApp.war

Cernunnos & uPortal

New or Improved Features

Cernunnos has been used with uPortal in

several ways:

• Execute arbitrary SQL from the console

• Convert a new uPortal 2.5.x to DLM

• Improved “deployPortletApp” target

• Core data Import/Export (XML-based)

Execute Arbitrary SQL

• Cernunnos picks up JDBC connection information

from rdbm.properties (viz. where it lives)

• Invoke with

> crn sql.crn “INSERT INTO up_user…”

<properties location="build/WEB-INF/classes/properties/rdbm.properties"> <sql-connection

driver="${req(jdbcDriver)}" url="${req(jdbcUrl)}" username="${req(jdbcUser)}“password="${req(jdbcPassword)}">

<sql-statement sql="${req($1)}"/> </sql-connection></properties>

Convert to DLM

• Automates the process described on the uPortal wiki:

http://www.ja-sig.org/wiki/display/UPM/DLM+Administration+Guide

• Summary:

1. Set UserLayoutStoreFactory.implementation to RDBMDistributedLayoutStore

2. Set UserLayoutManagerFactory.coreImplementation to DistributedLayoutManager

3. Update the UP_USER_PROFILE table to match the theme and style values for DLM

Convert to DLM (Cont.)

<properties location="build/WEB-INF/classes/properties/rdbm.properties">

<!-- First change portal.properties... --> <string-replace string="${url(properties/portal.properties)}" regex="...UserLayoutStoreFactory.impl=..." replacement="...RDBMDistributedLayoutStore"> <string-replace regex="...UserLayoutManagerFactory.impl=..." replacement="...DistributedLayoutManager"> <write-file file="properties/portal.properties"/> </string-replace> </string-replace>

<!-- Next update the database... --> <sql-connection ...> <sql-statement sql="UPDATE up_user_profile

SET structure_ss_id=4, theme_ss_id=3 WHERE theme_ss_id=11"/>

</sql-connection>

</properties>

New “deployPortletApp” Target

• Current deployPortletApp implementation creates a

new web.xml file using some values from the existing

one, which causes problems

– Hard-coded use of the 2.3 servlet spec (UP-1241)

– //description becomes /web-app/description

• New implementation merely adjusts the web.xml that is

provided

• New implementation also makes it more natural to

invoke this behavior within the web application itself

classpath://org/jasig/portal/container/deploy/deploy-portlet-app.crn

New “deployPortletApp” Target (Cont.)

Summary of deployPortletApp behavior:

1. Extract the specified portlet WAR file to the appropriate location within Tomcat (deploy.home in Ant)

2. Add <servlet> elements for each portlet definition in portlet.xml using XSLT

3. Add <servlet-mapping> elements for each <servlet> element using XSLT

4. Add a <taglib> element for portlet JSP tags

5. Add the portlet.tld file to the webapp in Tomcat

XML Import/Export

• XML Import & Export using Cernunnos was

presented at the uPortal Developers’ Meeting at

Johns Hopkins University in April

• This portion of the session contains updated

material from that meeting

• Much more detailed information -- and a working

prototype -- are available onlinehttp://www.ja-sig.org/wiki/display/UPC/uP2+XML+Import+Export+with+Cernunnos

XML Import/Export (Cont.)

• Although every uPortal release comes with a

working data set, every institution needs to

customize this data extensively to meet their

individual needs

• Editing this data where it lives (i.e. data.xml)

is confusing, tedious, and risky!

• Using uPortal’s UI to edit the data isn’t

reproducible by automated means

XML Import/Export (Cont.)

It would be nice if…

• The data set were more approachable

(readable and editable)

• Entities were described in individual files

instead of a single, monolithic document

• Entities didn’t reference each other using

database IDs, so they could be imported (or

shared) in any order

Example XML Document

<layout username="student" script="classpath://org/jasig/portal/layout/simple/import-

layout.crn"> <root name="Root folder" immutable="N" unremovable="Y"> <header name="Header folder" immutable="Y" unremovable="Y"> <channel fname="header"/> <channel fname="portal/login/general"/> </header> <tab name="Main Student Tab" immutable="N" unremovable="N"> <column name="Column 2" immutable="N" unremovable="N"> <channel fname="salon.com"/> </column> <column name="Column 1" immutable="N" unremovable="N"> <channel fname="minesweeper"/> <channel fname="word-of-the-day"/> <channel fname="number-guessing-game"/> </column> </tab> <footer name="Footer folder" immutable="N" unremovable="N"> <channel fname="footer"/> </footer> </root></layout>

Using the Prototype

• A Cernunnos-based import/export tool is available that handles the following data:

– Users

– Groups

– Channels

– Group Memberships

– Permissions

– Layouts

• The prototype tool is attached as a ZIP to the wiki page about this technology

• To use it, extract the contents into the root of a uPortal 2.5 (or later) distribution

Using the Prototype (Cont.)

• Invoke it from the command line using Ant

> ant –f import-export.xml export –Ddir={dir} –Dtype={type} [–Dsysid={sysid}]

> ant –f import-export.xml import [-Ddir={dir}] [-Dpattern={pattern}]

dir = a file system directory

type = all | layout | all-layouts | channel | all-channels | all-permissions | all-memberships | group | all-groups

sysid = user_name | fname |group_name

pattern = a regex expression

Andrew [email protected]

code.google.com/p/cernunnos/

Questions?

** REMOVED **

Custom Software is Expensive

• Developing new software using "industry

standard" technologies and practices is very

effort intensive

• Qualified professionals ("knowledge workers")

are expensive

• And don’t forget…

– Software project failure rates are daunting

– So are time-to-market considerations

Whose Problem Is It?

• If you're not thinking about these issues every day, you really should... your employers are

• Offshoring is a popular practice, and growing in use

• Many valuable software projects are never undertaken

• There is a high threshold of expected business value that a proposed software project must meet

Choosing Project Scope

One common approach to planning the scope of a

software project and its iterations:

1. Create a list of proposed features

2. Assign each feature a number for Business Value

3. Assign each feature a number for Effort

4. Sort the list by the ratio of Business Value to Effort

5. Plan to implement the features with the best ratio first

6. Stop when the incremental value no longer justifies the

effort

Choosing Project Scope (Cont.)

Feature BV/LOE LOE BV

A 1.33 3 4

B 1.25 4 5

C 1.20 5 6

D 0.78 9 7

E 0.75 8 6

F 0.71 7 5

G 0.67 6 4

H 0.57 14 8

I 0.54 13 7

J 0.50 12 6

K 0.45 11 5

L 0.45 20 9

M 0.42 19 8

N 0.40 10 4

O 0.39 18 7

P 0.35 17 6

Q 0.31 16 5

R 0.27 15 4

S 0.23 22 5

T 0.19 21 4

Relationship Between Business Value & Effort

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

A B C D E F G H I J K L M N O P Q R S T

Proposed Features

BV/LOE

Choosing Project Scope (Cont.)

It Doesn’t Have to Be This Way

• Among human endeavors, software is

exceedingly young

• Software industry technologies and practices

are at an early evolutionary stage

• Innovations that unlock tremendous gains in

effectiveness are out there to be invented

Relationship Between Business Value & Effort

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

A B C D E F G H I J K L M N O P Q R S T

Proposed Features

BV/LOE

Choosing Project Scope (Revisited)

Goal of This Session

Buzzword Compliance

• Cernunnos leverages forward thinking both

from within the software industry and

elsewhere

– Composite Pattern

– Transparent Enclosure/Decorator Pattern

– Behavior Injection

– Intelligent Defaults

– Form Follows Function

– Platform Independence

Solution

• Storing the data in XML is great because

tools like Subversion can manage it smoothly

• But the XML format used by data.xml is

unapproachable to human beings

• It would be better to store this data in a

collection of smaller XML documents with

domain-specific schemas

Solution (Cont.)

• These documents can be more succinct and

manageable by conveying information

structurally that data.xml must convey

explicitly (e.g parent-child relationships, entity

types)

• Also these documents can avoid referencing

database IDs, allowing entities to be imported

and exported ad hoc – even from one portal

database to another!

Data in uPortal

• Information in the portal database represents the following classes of entities:

– Minor Items (version info., sequences, etc.)

– Users

– Groups

– Channels

– Group Memberships

– Permissions

– Layouts

• Items that appear lower on this list often depend on items that appear higher (viz. foreign keys)

Status of the Prototype

• Work on the prototype started with Layouts (DLM) and proceeded “backwards”

• The prototype currently includes both import and export through Groups Users

• Which leaves:

– Minor Items

– Users

• It works in uPortal 2.5.3 (AFAIK)

• It hasn’t been tried in any other version