3 the system architecture

BEXIS Tech Talk Series

#3: The System Architecture

Javad ChamanaraMarch 2016

BEXIS Tech Talk #3: The System Architecture 2

Recall from the first talk

• Requirements– Data Lifecycle Management– Generic– Extensible– Portable– Scalable


Requirements -> DLM

• Flexible Data Structures• Data Submission• Validation• Preserving• Metadata Management• Versioning


Recall from the second talk

DataMetadata

Data StructureMetadata Structure Semantics Geo

Administration Security

«use»

«use»

«use» «use»

«use»


Core Conceptual Model

SearchPublishing

CMLand Use

ReservationData Submission


DLM APIs

Core Functions

Data Structure Mgmt.Data Mgmt.

Metadata Mgmt.Sem. Data Mgmt.


Multi DBMS data access

DB2 PgS

Data Access

Core Functions

...

RelationalXMLGIS

Multi DBMS supportVendor unlocking

ORM supportCustomization support

Data Structure Mgmt.Data Mgmt.

Metadata Mgmt.Sem. Data Mgmt.


Access to the Core Functions

DB2 PgS

Data Access

Core Functions

UI Framework

...

Modularity

Integration


Access to Core Functions

DB2 PgS

Data Access

Core Functions

UI

UI Framework

...

Modularity

IntegrationSynthesis Work

Semantic Search

Analytics

User Defined

Spatial Querying

External Tools

Web Services

Archiving

Import/ Export

Publishing


Security

DB2 PgS

Data Access

Security

Core Functions

UI

UI Framework

...M

odularity

IntegrationSynthesis Work

Semantic Search

Analytics

User Defined

Spatial Querying

External Tools

Web Services

Archiving

Import/ Export

Publishing


Architectural Patterns

• Generic User Interaction• MVC• APIs, Web APIs• Data Access• Services


Generic User Interaction

Presentation

User

Services Core Services Security Data Access


Presentation: MVC

class mvc

Controller

ModelView

«observe»

«observe»


Simplified MVC

class mvc

View ViewModel

Controller

«observe»

Dependency


Classes Engaged in an Interaction

Controller

View

Service Framework API

Data

GIS Data

ViewModel

«Observe»

«use»

«use»


BExIS MVC Sequence Diagram

User

Controller Model ServiceViewRouting Engine

Request()

action()

GetData()

Populate()

Bind()

Render()


MVC Constraints

• Passive Models• Request more data• Manipulate entities

• Models are POCOs• Models have no access to services• Models may have builder/ transformers• Views can interact with controllers’ actions only

– Interchanging data– Calling controllers’ actions either sync or async

• No lazy loading in the Views (populated models+ajax)


Controller

View

Service Framework API

Data

GIS Data

ViewModel

«Observe»

«use»

«use»

Presentation

Data

Business


Integration

• Service Based– Modules provide a set of services

• HTTP/S + Auth– Internal authentication– Remote LDAP/ Active Directory integration

• Authorization– Restricts access to controllers’ actions– Public actions– Public data


Public APIs

• Web API– API namespace

• Module/Controller/Action• Verb• Data

• www.example.com/dcm/datasets/1• www.example.com/dcm/datasets/1/versions• www.example.com/dcm/versions/10• Querystrings are allowed• No versioning on APIs! Always the latest!!

http://www.example.com/dcm/datasets/1

http://www.example.com/dcm/datasets/1

http://www.example.com/dcm/versions/10


Modularity

• Modules– A set of related user facing functions– To extend the application’s functionality

• Layering– Presentation

• MVC– Service

• Access to Core services, DLM, external services, etc.– Data Access

• Own Database/data• Direct access to the DB/data (not recommended)


Modularity

• Integration– Shell integration: menus, themes, scripts, etc.– Authorization

• Introducing actions, to be secured, to the system• Automatic security trimming/ action authorization

• Notes– Deployment– Hot plug-abilities– Upgrading/versioning


Data

• Relational data– Schema evolution– Schema generation– Data versioning– Vendor/tool independence

• XML data– File based XML– RDBMS based XML

• GIS data– External systems– Integrated querying– Security


Data Access

• ORM– NHibernate– Vaiona


Data Access Patterns

• Repository pattern– CRUD on entities– Querying using

• LINQ to objects• HQL• Native SQL (queries, views, stored-procedures)• Vendor specific customization

– Relationship traversal• Laziness/Eagerness• object graph traversal

– Automatic entity state/context management– Based on: http://martinfowler.com/eaaCatalog/repository.html

http://martinfowler.com/eaaCatalog/repository.html

http://martinfowler.com/eaaCatalog/repository.html



• Unit of Work pattern– Business Transaction Management– Concurrency Control– Operating on multiple repositories– Transactions

• Distributed Transaction Coordination• Transparent transaction management• Ambient transaction detection/reuse

– Web request/session/user aware transaction mgmt.– Automatic commit on request/session termination– Based on: http://martinfowler.com/eaaCatalog/unitOfWork.html

http://martinfowler.com/eaaCatalog/unitOfWork.html

http://martinfowler.com/eaaCatalog/unitOfWork.html



• Aggregates– Cluster of entities and their dependents

• Datasets and their Versions• Units of measurement and their conversion formula

– Simplify the service layer– Packaged as units of work– Better business level operation/transaction mgmt.– Look at: http://

martinfowler.com/bliki/DDD_Aggregate.html

http://martinfowler.com/bliki/DDD_Aggregate.html

http://martinfowler.com/bliki/DDD_Aggregate.html



• Materialization– Complex entity serialized as XML– Stored in field of a table row– Materialize on read

• Reads the XML and loads the entity– Dematerialize on write

• Serializes the object as an XML document– Used in Data Tuples of dataset versions


ORM example

• Dataset as an Aggregate area• DataTuple• DataCell• Versions• Associations


Dataset aggregate: conceptual diagram

Dataset

Dataset VersionDataset Stage

Tuple

DataValueData Cell

Data Structure

1

+Current Stage

1

10..1

1

1+Variable Values

{Delta Association,Only Structured Data}


Dataset aggregate: class diagram


General Record Structure

Identification

Payload Associations

Versioning Audit State Technical Extra



Identification



Identification: it holds information about primary keys, key generation, etc.



Identification



Payload: is the business data that the table holds



Identification



Associations: relationships between this table and others (also itself). Connected and not connected foreign keys fall into this category, too.



Identification



Versioning: information about version number and timestamp. This is the technical data level versioning to be used in concurrency control. It is not the business level data versioning.



Identification



Audit: the information about the person or the system who has changed the data in addition to the time, action, and other information



Identification



State: shows the current status of the data record. It can be set by business workflows or quality control, e.g., “Approved”, “Rejected”, “Published”, “Locked”, etc.



Identification



Technical: this part is used for development purposes and is not intended to be used by data owners or data consumers



Identification



Extra: It is an XML field to provide a place for minor extensions and reduce the amount and frequency of changes in the database schema (and sometimes data). Also, it can be considered as an exertion point for customization in different scenarios.


ORM mapping: The entity

<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="BExIS.Dlm.Entities" namespace="BExIS.Dlm.Entities.Data">

<class xmlns="urn:nhibernate-mapping-2.2" name="Dataset" table="Datasets">

</class>

</hibernate-mapping>


ORM mapping: Non payload attributes

<id name="Id" type="Int64"> <column name="Id" /> <generator class="native" /></id><version name="VersionNo" type="Int32"> <column name="VersionNo" /></version> <property name="Extra" type="System.Xml.XmlDocument, System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"> <column name="Extra" not-null="false" sql-type="xml" /></property>


ORM mapping: State attributes

<component name="StateInfo"> <property name="State" type="string"> <column name="State" /> </property> <property name="Timestamp" type="DateTime"> <column name="StateTimestamp" not-null="false"/> </property> <property name="Comment" type="string"> <column name="StateComment" />

</property></component>

Same for Creation Info and Modification Info


ORM mapping: Payload attributes

<property name="LastCheckIOTimestamp" type="DateTime"> <column name="LastCheckIOTimestamp" /></property> <property name="CheckOutUser" type="string"> <column name="CheckOutUser" /></property>


ORM mapping: Associations

<set name="Versions" inverse="true" lazy="true" order-by="Timestamp desc" cascade="all-delete-orphan"> <key> <column name="DatasetRef" /> </key> <one-to-many class="DatasetVersion" /></set>


Dataset aggregate: Database Schema


How to work with the model

DatasetManagerService dms = new DatasetManagerService();

Dataset ds = dms.CreateEmptyDataset(sds, rp, ms);

DatasetVersion workingCopy = dms.GetDatasetWorkingCopy(ds);

// edit the version

dms.CheckInDataset(ds, “new version is created.", user);


Inside the “Check In” service method

public void CheckInDataset (Int64 datasetId, string comment, string userName) { using(IUnitOfWork uow = this.GetUnitOfWork()) { IRepository<Dataset> repo = uow.GetRepository<Dataset>(); Dataset ds = repo.Get(…).FirstOrDefault();

// Check-in logic here… repo.Put(ds); uow.Commit(); } } }


Outlook

Whats next in the talk series?• General purpose components• Third party components• Cross-cut functionalities


Thanks!Questions?

Contact:[email protected]://fusion.cs.uni-jena.de/bexis

Acknowledgment

3 the system architecture

Software