Middleware Early Adopters Report: Technical Implementations

Middleware Early Adopters Report:Technical Implementations

31 October 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 2

Panelists

• Tom Barton, University of Memphis

• Louise Miller-Finn, Johns Hopkins University

• Jack Suess and Rob Banz, University of Maryland, Baltimore County

• Moderator: Ken Klingenstein, Internet2 /University of Colorado

Early Adopters Middleware Reports

Technical Implementation

The University of Memphis

Dr. Tom Barton

Internet2 Fall 2000 Meeting: Early Adopters Report 4

Initial Middleware Architecture

HRS

SIS

UUIDs

1

Ph Rebuild& AMP

BuildUser

cardswipe

WebPh

2

rsh utils

SMTProuting

Dialup

Labs &Desktops

Addressbooks

HTTPauthen

HTTPauthor

IMAPemail

HTTPemail

Calendar

qiSynch

NDSv5

DSPh

3

UMDI

Internet2 Fall 2000 Meeting: Early Adopters Report 5

Project Scope

• Replace or reengineer Ph applications• Enhance existing data feeds from administrative systems

• New metadirectory solution and registry process

• Group messaging & group authorization facilities

• New account maintenance facilities & practices• Prepare for next stage: PKI and Roles

Internet2 Fall 2000 Meeting: Early Adopters Report 6

Final Middleware Architecture

HRS

SIS

UUIDs

1

Registry &AMP

Acct MaintWeb Site

Ph<->DSShim

SMTProuting

RADIUS

Labs &Desktops

Addressbooks

HTTPauthen

HTTPauthor

IMAP/HTTPemail

CMS/Portal

Calendar

NDSv8

DS

SMTPAUTH

DSGW

Groupmessaging

AD

2

ChangeDispatcher

Misc PhClients

FRS

PAM/NSS

OtherActions

Internet2 Fall 2000 Meeting: Early Adopters Report 7

LDAP SMTP Routing

• Two phases: Identification & Routing

• Routing issue: when to rewrite envelope recipient

• One solution: “LDAP routing domain” conceptDefinition: all MTAs that use the same set of LDAP

searches against the same directory for the same set of SMTP domains to make email handling decisions.

Guideline: Entries whose mailbox resides on a mailbox host inside the LDAP email routing domain should only have mailHost populated. Other entries should only have mailRoutingAddress populated. With this guideline, it is unnecessary to ever have both attributes populated.

Internet2 Fall 2000 Meeting: Early Adopters Report 8

LDAP SMTP Routing (cont’d)

Attribute Idx Description

mail eq Displayed email address.

mailAlternate-Address

eq Additional addresses by which this entry is known. Multi-valued.

mailHost none Mailbox hostname (if within the same LDAP routing domain).

mailRouting-Address

none rfc822mailbox, provided that the mailbox host lies outside of the LDAP routing domain.

Internet2 Fall 2000 Meeting: Early Adopters Report 9

LDAP SMTP Routing (cont’d)

That complicated real world legacy:

• Old “Ph aliases” still had to be deliverable.

• Fallback routing methods embodied in custom extensions to phquery

• Aliases files on mailbox hosts being retired

• “Vanity” email domain addressing

Put legacy addresses in mailAlternateAddress

Analyze mail logs proactively to catch legacies

Leave “passthru mode” enabled for a good while

Internet2 Fall 2000 Meeting: Early Adopters Report 10

Group Messaging & Authorization

Populate LDAP group objects to support authorized access to email distribution lists & web resources.

• Top-down; majors; class; section; org related

• LDAP backed SMTP AUTH over TLS used to identify poster’s uid. LDAP (uid=) search to find poster’s DN, which is compared to a list of permitted DNs for each mail group.

• Uses Netscape Messaging Server’s mailGroup objectclass for its “allowedBroadcaster” attribute.

Internet2 Fall 2000 Meeting: Early Adopters Report 11

Representing Organizational Structure

Labelled organizational hierarchy tree structure from:• FRS responsibility rollup• FRS DBD• Ad hoc rules to prune or identify certain nodes and to create realistic labels (the hardest part)

Payroll data is then used to assign each employee to one or more nodes in the tree.

FRS security file indirectly identifies dept heads at each node.

Tree is mapped to ou=orgUM branch of DIT and org-related attributes of people objects.

Internet2 Fall 2000 Meeting: Early Adopters Report 12

Representing Organizational Structure (cont’d)

ou=Anthropology, ou=College of Arts & Sciences, ou=Provost, …

umOrgDeptHead: <DN> single-valued

umOrgBudgetHead: <DN> multi-valued

umOrgMemberGroupDN: <DN of groupOfUniqueNames>

umOrgRollupMemberGroupDN: ditto

umOrgRollupDeptHeadsGroupDN: ditto

umOrgRollupBudgetHeadsGroupDN: ditto

People attributes:

ou: <list of ouRDNs>

umRollupID, umSuperiorRollupID: <FRS rollup IDs> single-valued

Or labeledURI in

searchGuide?

Internet2 Fall 2000 Meeting: Early Adopters Report 13

Representing Roles & Sets of Correlated Information

Example: jdoe is faculty in Law & staff in Econ. Don’t want jdoe when searching for faculty in Econ.

role: {-affiliation:faculty-dept:law, -affiliation:staff-dept:Econ}

roleDN: {DN of role1, DN of role2}

cn=role1, ou=Roles, …

roleDept: Law

roleAffiliation: Faculty

…

cn=role2, ou=umRoles, …

roleDept: Econ

roleAffiliation: Staff

Early Adopters Middleware Reports

Technical Implementation

Johns Hopkins University

Louise Miller-Finn

Internet2 Fall 2000 Meeting: Early Adopters Report 15

Johns HopkinsJohns Hopkins

Directory Data Flows

•Establish methodology for populating

the directory

•Select vendor products and tools

•Establish standards

•Staff the team

•Use iterative development cycle

JHUPayroll

JHU USIS

__________

ID Badging

__________

D atabase Load

R egistry(O racle)

Back EndBusiness R ulesand P rocessing

Prepare andexport da ta fo r

d irectory

D irectory F iles

Load EnterpriseD irectory

EnterpriseD irectory(iP lanet)

IngeniumD atabase

W eb Enabled T&EFront End

Authentication

Identify Data sourcesneeded for front endauthentication;

D efine andim plem entbusiness ru les fo rthe T&Eapplication;

D efine theattribu tes neededfor the T&E frontend;

P rovide front endbusiness ru lesand userrequirem ents topresent andcapture data ;

D efine thedatabaserequirem entshere and m ap tothe ESG O racleG old Source --ESG D irectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

Stage 1 S tage 2 S tage 3 S tage 4 S tage 5

D irectory S ta tusV iew

S tage 6

S tage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 17

Johns Johns HopkinsHopkins

Stage 1 – Analyze Data Sources•Identify Data Sources

• Where do the data feeds originate; what data fields are required;

• Provide Standard Data Collection Model

• What is the frequency of the data feed; require fixed length fields and records;

•Define database load procedure

•Produce audit log

JH U Payro ll

JH U U SIS

__________

ID Badging

__________

DatabaseLoad

R egistry(O racle)

Back EndBusiness R ulesand P rocessing

Prepare andexport da ta fo r

d irectory

D irectory F iles

Load EnterpriseD irectory

EnterpriseD irectory

IngeniumD atabase

W eb Enabled T&EFront End

Authentication

Identify D atasources neededfor fron t endauthentica tion;

D efine andim plem entbusiness ru les fo rthe T&Eapplication;

D efine theattribu tes neededfor the T&E frontend;

P rovide front endbusiness ru lesand userrequirem ents topresent andcapture data ;

D efine thedatabaserequirem entshere and m ap tothe ESG O racleG old Source --ESG D irectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

S tage 1 Stage 2 S tage 3 S tage 4 S tage 5

D irectory S ta tusV iew

S tage 6

S tage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 19

Johns HopkinsJohns Hopkins

Stage 2 – Database Requirements

•Define the input tables to represent the clients’ data; define key fields to tie tables together;

•Provide data model using an Entity Relationship tool (e.g. ERWin);

•Document and store common database procedures;

•Provide standard database templates for reuse;

•Provide audit log

JH U Payro ll

JH U U SIS

__________

ID Badging

__________

D atabase Load

Registry(Oracle)

Back EndBusiness Rulesand Processing

Prepare andexport data for

d irectory

D irectory F iles

Load Enterrp iseD irectory

EnterpriseD irectory(iP lanet)

IngeniumD atabase

W eb Enabled T&EFront End

Authentication

Identify D atasources neededfor front endauthentica tion;

Define andimplementbusinessrules

D efine theattribu tes neededfor the T&E frontend;

Provide front endbusiness ru lesand userrequirem ents topresent andcapture data;

D efine thedatabaserequirem entshere and m ap tothe ESG O racleG old Source --ESG D irectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

S tage 1 S tage 2 Stage 3 S tage 4 S tage 5

D irectory S tatusV iew

S tage 6

S tage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 21

Johns HopkinsJohns Hopkins

Stage 3 – Back End Processing

•Develop procedures (PLSQL) to process high level business rules;

•Implement ER diagrams that define table fields;

•Create intermediate tables with directory records;

•Store common procedure templates for reuse;

•Provide audit log

JH U Payro ll

JH U U SIS

__________

ID Badging

__________

D atabase Load

R egistry(O racle)

Back EndBusiness R ulesand P rocessing

Prepare andexport datafor directory

DirectoryFiles

Load EnterpriseD irectory

EnterpriseD irectory(iP lanet)

IngeniumD atabase

W eb Enabled T&EFront End

Authentication

Identify D atasources neededfor fron t endauthentica tion;

D efine andim plem entbusiness ru les fo rthe T&Eapplication;

Define theattributesneeded forthe frontend;

Provide front endbusiness ru lesand userrequirem ents topresent andcapture data ;

D efine thedatabaserequirem entshere and m ap tothe ESG O racleG old Source --ESG D irectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

S tage 1 S tage 2 S tage 3 Stage 4 S tage 5

D irectory S ta tusV iew

S tage 6

S tage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 23

Johns HopkinsJohns Hopkins

Stage 4 – Database Export

•Create export file in fixed field, fixed record format;

•Develop status field processing using eye catcher (e.g. ‘ADD’, ‘DELETE’, ‘UPDATE’, ‘NOCHANGE’)

•Document export procedure and standard field values;

•Create and store common export procedure template;

•Produce activity log

JH U Payro ll

JH U U SIS

__________

ID Badging

__________

D atabase Load

R egistry(O racle)

Back EndBusiness R ulesand P rocessing

Prepare andexport da ta fo r

d irectory

D irectory F iles

LoadDirectory

EnterpriseDirectory

IngeniumD atabase

W eb Enabled T&EFront End

Authentica tion

Identify D atasources neededfor fron t endauthentica tion;

D efine andim plem entbusiness ru les fo rthe T&Eapplica tion;

D efine theattribu tes neededfor the T&E frontend;

P rovide front endbusiness ru lesand userrequ irem ents topresent andcapture data ;

D efine thedatabaserequirem entshere and m ap tothe ESG O racleG old Source --ESG D irectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

S tage 1 S tage 2 S tage 3 S tage 4 Stage 5

D irectory S ta tusV iew

S tage 6

S tage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 25

Johns HopkinsJohns Hopkins

Stage 5 – Directory Import

•Process export files using generic (PERL) script to import/update enterprise directory;

• Keep code free of business rules;

•Create and store common script template for reuse;

•Provide web based report interface to track activity and status;

•Provide audit log

Directory Structure

Registry EnterpriseDirectory

SourceSystem Data

Feeds

DirectoryUpdates

Source data isloaded into thestaging area andassigned aunique identifier;

The directoryupdate usesthe uniqueidentifier asthe key link forall entries inthe variousbranches ofthe directory;

Authentication Branch(holds unique identifier (UID) for

person)

______(keyed by

UID)

admin branch(keyed by

UID)

library patron(keyed by

UID)

jh_userid(keyed by

UID)

jh_person(keyed by UID)

Web EnabledFront End

Application

For example:user provideslast name tosearch, and abarcode/PIN toauthenticate tothe patronlibrarydirectorybranch;

The UID isderived fromthe OracleRepository as aprimary key;uses a JH ISOnumberscheme;

JH U Payro ll

JH U U SIS

__________

ID Badging

__________

D atabase Load

R egistry(O racle)

Back EndBusiness R ulesand P rocessing

Prepare andexport da ta fo r

d irectory

D irectory F iles

Load D irectory

EnterpriseDirectory

IngeniumD atabase

W eb EnabledFront End

Authentication

Identify D atasources neededfor fron t endauthentica tion;

D efine andim plem entbusiness ru les fo rthe application ;

D efine theattribu tes neededfor the front end;

P rovide front endbusiness ru lesand userrequirem ents topresent andcapture data ;

D efine thedatabaserequirem entshere and m ap tothe R eg istry --EnterpriseD irectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

S tage 1 S tage 2 S tage 3 S tage 4 S tage 5

DirectoryStatus View

Stage 6

S tage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 28

Johns HopkinsJohns Hopkins

Stage 6 – Directory Status

•Provide complete audit log of directory

activity;• Create and store common report template;

• Generate standard web based activity report;

•Provide backup/recovery procedure;

•Provide replication service

JH U Payro ll

JH U U SIS

__________

ID Badging

__________

D atabase Load

R egistry(O racle)

Back EndBusiness R ulesand Processing

Prepare andexport data for

d irectory

D irectory F iles

Load D irectory

EnterpriseDirectory

IngeniumDatabase

W eb EnabledT&E Front EndAuthentication

Identify D atasources neededfor front endauthentica tion;

D efine andim plem entbusiness ru les forthe T&Eapplication;

D efine theattribu tes neededfor the T&E frontend;

Provide frontend businessrules anduserrequirementsto presentand capturedata;

Define thedatabaserequirementshere andmap to theRegistry --EnterpriseDirectory;

D irectory U pdateLog to D ata

Sources

D irectoryU pdateH old ing

Area

S tage 1 S tage 2 S tage 3 S tage 4 S tage 5

D irectory S tatusV iew

S tage 6

S tage 8

Stage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 30

Johns HopkinsJohns Hopkins

Stage 7 – Front End Processing•Define and deploy access control (ACL);

• Define JHI policy for the global user, the person, and the administrator;

• Develop and document scope and visibility to the directory attributes;

•Develop and deploy common web enabled directory access (a common ‘look and feel’ to the front end);

• Use a common set of development tools (e.g. ColdFusion);

•Apply front end application level business rules (more specific rules than the back end process);

JHU Payroll

JHU USIS

__________

ID Badging

__________

D atabase Load

R egistry(O racle)

Back EndBusiness R ulesand Processing

Prepare andexport data for

d irectory

D irectory F iles

Load D irectory

EnterpriseDirectory

IngeniumD atabase

W eb EnabledFront End

Authentication

Identify D atasources neededfor front endauthentica tion;

D efine andim plem entbusiness ru les forthe T&Eapplication;

D efine theattribu tes neededfor the T&E frontend;

Provide frontend businessrules anduserrequirementsto presentand capturedata;

D efine thedatabaserequirem entshere and m ap tothe R egistry --EnterpriseD irectory;

DirectoryUpdate Log toData Sources

DirectoryUpdate

Holding Area

S tage 1 S tage 2 S tage 3 S tage 4 S tage 5

D irectory S tatusV iew

S tage 6

Stage 8

S tage 7

Johns H opkins Insititu tionsO ctober 2000

Internet2 Fall 2000 Meeting: Early Adopters Report 32

Johns HopkinsJohns Hopkins

Stage 8 – Directory Updates

•Provide a log dataset of directory activity

(updates, deletes, etc.);

•Provide standard procedure for data owners

to pull the activity log;

•Design and implement a standard record

layout using a status field and an audit trailer

record;

Internet2 Fall 2000 Meeting: Early Adopters Report 33

Johns HopkinsJohns Hopkins

Iterative Development Cycle

•Provide complete data-to-directory path;

•Push the data through one cycle to kickoff development process;

•Each iteration flushes more detail to the requirements in a rapid application development process adding data, business rules and/or policy changes;

•Each iteration provides intense unit testing followed by QC test cycle;

Early Adopters Middleware Reports

Technical Implementation

University of Maryland

Baltimore County

Robert Banz

Internet2 Fall 2000 Meeting: Early Adopters Report 35

UMBC – Present Architecture

Internet2 Fall 2000 Meeting: Early Adopters Report 36

UMBC – Hardware & Software

•Hardware• 1 Sun Enterprise 220 (2x CPU, 2G RAM) – Primary Directory Server

• 2 Sun NetraT1 – Slave Directory Servers

•Software• iPlanet (Netscape) Directory Server

• Loads and loads of Perl

Internet2 Fall 2000 Meeting: Early Adopters Report 37

UMBC – Person Directory

• Contains over 275,000 Entries from:• Human Resources -- in Oracle

• SIS – Resides on an HP MPE machine, data is mirrored “almost real time” into Oracle

• PH/CSO Database Contents (retired)

• “Other” entities not contained in either of these data sources

• Not (yet) the System of Record

Internet2 Fall 2000 Meeting: Early Adopters Report 38

UMBC – Directory Schema

• “Flat” person directory

• Uses “umbcPerson” objectclass• Derived from InetOrgPerson

• Proposed “eduPerson” attribute names & definitions used when possible.

• Account management data stored in RFC2307 (NIS Objects in LDAP) compliant schema.

Internet2 Fall 2000 Meeting: Early Adopters Report 39

UMBC – Data Synchronization

• Currently “One-Way”

(changes must still be made through legacy applications and myUMBC web portal)

• Changes to HR & SIS data are written to a log table via PL/SQL triggers

• Perl daemon constantly checks the log table and takes appropriate action

Internet2 Fall 2000 Meeting: Early Adopters Report 40

UMBC – Directory Enabled Applications

•WebAdmin• UNIX Account Management (self service and

administrative access)• Person Directory Management• Email Namespace Management

•MyUMBC• Uses directory to resolve “people” from “usernames”

•@umbc.edu EMail Addresses• Destination addresses of all @umbc.edu are resolved

via a LDAP map in Sendmail.• Replaced PH/CSO “phquery”

Internet2 Fall 2000 Meeting: Early Adopters Report 41

UMBC – Future Applications

•@umbc.edu Email Addresses for Alumni

•Tighter integration of On-Line course tools (WebCT, Blackboard), including:

• Automagic enrollment of students in registered courses

•“Single Sign On” to campus Web services

•Integration with PeopleSoft• Most likely will become the “Registry of Record”

Internet2 Fall 2000 Meeting: Early Adopters Report 42

For More Information

• www.internet2.edu/middleware/earlyadopters/

• University of Memphis–Tom Barton tbarton@memphis.edu

• Johns Hopkins University–Louise Miller-Finn lmiller@jhmi.edu

• University of Maryland, Baltimore County–Jack Suess jack@umbc.edu