Building an Integration Platform with Gradle/Grails - Spring 2GX 2012

© 2012 ADAPTIVE COMPUTING, INC. 1

Building an Integration Platform with Grails and Gradle

Brian Saville

Adaptive Computing

18 Oct 2012

© 2012 ADAPTIVE COMPUTING, INC. 2 © 2012 ADAPTIVE COMPUTING, INC. 2

Me

Brian Saville

[email protected]

Twitter: @bksaville

Plugins:

▪ External Config Reload

▪ Spring Security OAuth2 Provider

▪ New Doc

▪ Artefact Messaging

▪ Contributor on Others


Adaptive Computing Highlights

▪ Leaders in cloud and HPC software

▪ 10+ years developing Moab® technology

▪ Leadership team w/ $1.5B+ in exits

▪ 15 issued patents, 34 pending

▪ Backed by top-tier investors:

▪ Global Partnerships: HP, IBM, SGI, Cray, Microsoft, etc.

▪ Over 200 customers such as:

http://www.linkedin.com/groups?home=&gid=1846591&trk=anet_ug_hm

http://www.epicvc.com/


Moab Web Services

▪ What is it?

▪ Grails 2.0.3

▪ RESTful Web Services

▪ Business Logic

▪ Plugin Framework (Beta)

▪ What does it do?

▪ Wraps C Core Product

▪ Integrates “old world” non-standardized API with “new world” APIs


Atypical Grails Deployment

▪ MWS Deployed on Client Servers

▪ WAR file

▪ Tomcat 6

▪ Oracle/Sun JDK 6

▪ MongoDB 2.0+

▪ Customization on Client Sites


Integration Platform

▪ Reasons

▪ Extensibility

▪ In-House Developers

▪ External Developers

▪ Dynamic Updates to Application Behavior

▪ Why Create Our Own?

▪ Consists of Plugin Framework (Grails) and SDK (Gradle)



Why Do We Care?

▪ Integration Platform Developers

1) Provides a standard pattern to follow

2) Identifies required and optional framework components

3) Treads ground for new integration platforms

4) Identifies hurdles and potential problems

▪ Grails/Gradle Users

1) New advanced application of Grails and Gradle

2) Grails plugin framework can be evaluated with the pattern

3) Gradle plugin framework can be evaluated with the pattern


Integration Platform Pattern


The Pattern

▪ Plugin Identification

▪ Lifecycle and State Management

▪ Dependency Management

▪ Extend Application

▪ Services

▪ Configuration Management

▪ Lifecycle Hooks

▪ Metadata

▪ Development (SDK)


Plugin Identification

▪ By Convention

▪ Class names

▪ Packaging (JAR, Java or Groovy file)

▪ By Interface(s)

▪ IPlugin

▪ Activator (OSGI)

▪ By Configuration

▪ MANIFEST.MF


Lifecycle and State Management

▪ Concept of State

▪ Stopped and started

▪ More states optional

▪ Control Over State

▪ How to stop, how to start?

▪ How to add plugin? (Removal not required)


Dependency Management

▪ Internal Dependencies

▪ Build process

▪ Consider IDE

▪ External Dependencies

▪ Fat JARs

▪ Dependency resolution when adding


Extend Application

▪ Different in Each Case

▪ Custom Services

▪ Web services

▪ Internal services

▪ Provide Resources

▪ Report or Persist Data

▪ …


Services

▪ Stateless Services

▪ Control services

▪ Logging

▪ Dependent Services

▪ Configuration

▪ Data persistence


Configuration Management

▪ Access Application Configuration

▪ (Optional) Access Private Configuration

▪ Preferences


Lifecycle Hooks

▪ State Transition Hooks

▪ Starting and stopping

▪ Before, after, and/or during transition?

▪ Other Hooks

▪ Setup and configuration

▪ How Are They Defined?

▪ Convention

▪ Interfaces (Configuration)


Development (SDK)

▪ Dependency Management

▪ Build with internal dependencies

▪ Include external dependencies

▪ Testing

▪ Unit

▪ Integration

▪ Deployment

▪ (Optional) RAD Tool


Metadata

▪ By Convention

▪ By Configuration

▪ Types of Metadata

▪ Project/Plugin information

▪ Author information

▪ (Optional) Configuration constraints

▪ (Optional) Dependency information


Non-Essential Pattern Elements

▪ Security

▪ Deployment System/Central Repository

▪ Inversion of Control (IoC) Container

▪ Data Persistence

▪ Individual to Plugin

▪ Application Aware

▪ Common Clients and Parsers

▪ XML/JSON


Non-Essential Pattern Elements Cont.

▪ Events – Built-In and Custom

▪ Decomposition of Logic

▪ Messaging (i18n)

▪ Startup/Shutdown Concerns

▪ Scheduled Jobs

▪ Inter-Project Dependencies

▪ Environment Handling


Grails Plugins

▪ What is Missing?

▪ Standardized configuration management

▪ Platform core is addressing this

▪ Framework is Mature


MWS Integration Platform


Introduction

▪ Plugin Type – Plugin Definition Class

▪ One per Class

▪ Stored as Groovy or JAR files

▪ Internally is a Grails artefact

▪ Plugin Instance

▪ Multiple instances for a single plugin type possible

▪ Internally is DB entry and Spring Bean

▪ Dynamic (Running Application)


Plugin Identification

▪ “Ends With” Class Names

▪ Plugin Types

▪ Translators

▪ Autowired Classes

▪ Spring Annotated Classes

▪ @Component

▪ @Autowired

▪ @Resource

▪ @Scope


Lifecycle and State Management

▪ States

▪ Started, Stopped, Paused

▪ Plugin Control Service

▪ Start, stop, pause, resume

▪ Adding Plugins

▪ User Interface

▪ REST API


Dependency Management

▪ SDK for Build Dependencies

▪ Grails JARs

▪ Spring JARs

▪ Support Fat JARs

▪ Dependencies Resolved Before Adding


Extend Application

▪ Reporting – Application Aware Data Persistence

▪ Expose Custom Web Services

▪ Methods by convention

▪ Almost full REST API

▪ Unsecured annotation

▪ Routed by URL mappings

▪ Built-In Events by Convention


Services

▪ Logging (Similar to Grails)

▪ Data Persistence Services

▪ Control Service

▪ Individual Datastore Service

▪ SSL Service - SSLSocketFactory

▪ not-yet-commons-ssl with PEM files

▪ In memory keystore

▪ Internal REST Service

▪ Request and response with optional attributes

▪ Returns object with response, easy access properties, parsed data


Configuration Management

▪ Application Configuration

▪ Grails application bean

▪ Plugin Private Configuration

▪ Metaclass injected “config” property


Lifecycle Hooks

▪ Implemented as Methods on Plugin Types

▪ beforeStart, afterStart, beforeStop, afterStop

▪ configure

▪ If Not Implemented, Inject Empty Methods

▪ Called from Control Service


Metadata

▪ Convention

▪ Plugin project compiled class included in JAR

▪ Ends with “Project”

▪ Configuration

▪ Properties on plugin project class

▪ =~ Grails Plugin Class


Metadata – Dynamic Configuration Constraints

▪ Configuration is Dynamic Map

▪ Utilize Grails Constraints and Conventions

▪ Redo min/max to remove value checking

▪ Add custom plugin constrained property (CP)

▪ Add custom plugin CP builder

▪ Add Custom Constraints

▪ Type (Class) – Infer if not specified

▪ Default value

▪ Required (nullable)

▪ Scriptable URL (exec, file, http)

▪ Customized validator with additional methods


i18n Messaging

▪ Message Property Files

▪ Included in JAR

▪ Loaded with individual message source per project

▪ Message Method Injected on Plugins

▪ Used to Resolve Constraint Errors

▪ Not quite multi-lingual


Development (SDK)

▪ Commons JAR

▪ Interfaces, classes, constants, annotations

▪ Testing JAR – TestFor

▪ Gradle Project

▪ Dependencies

▪ Testing and Commons

▪ Creates fat JARs (lib and resolved deps)

▪ Additional build tasks

▪ Handles i18n property files

▪ Cobertura code coverage reports

▪ Unit testing with Spock


Development (SDK) - Commands

▪ Gradle

▪ create-project -Pproject-name=MyProject

▪ create-plugin -Pplugin-name=sample.MyPlugin

▪ test

▪ upload-test -Pmws.url=http://localhost/mws

▪ (Eventually) Python CLI Wrapper

▪ mplugin create-project MyProject

▪ mplugin create-plugin sample.MyPlugin

▪ mplugin test

▪ mplugin upload --url=http://localhost:8080/mws


Non-Essential Pattern Elements Status

▪ Security - Yes

▪ Central Repository – No

▪ IoC Container – Spring

▪ Data Persistence – Individual Datastore and Application Aware

▪ Common Clients and Parsers – Client Decided and Groovy

▪ Events – Built-In Only

▪ Decomposition – IoC with Conventions and Annotations


Non-Essential Pattern Elements Status Cont.

▪ Startup/Shutdown Concerns – Initial Plugins

▪ Scheduled Jobs – Quartz

▪ Inter-Project Dependencies – No

▪ Environment Handling – Application Configuration Parameter


Comparison to Grails Plugins

Concept MWS Plugins Grails Plugins

RAD Tool Partial X

Configuration X No Standard

Integration Tests X

Multiple Instances

X

Custom REST Partial X

Metadata Partial X

Languages Groovy Groovy & Java

Dynamic X

Lifecycle X X

i18n Partial X


Demo


Problems Encountered


JVM ClassLoaders

▪ Separating ClassLoaders

▪ Locking JARs

▪ Fat JARs

▪ Unresolved: Reloading Typed References


Language Support

▪ Groovy – Works Great

▪ Java – Interfaces and Metaclass Issues


Questions?

Building an Integration Platform with Gradle/Grails - Spring 2GX 2012

Technology

adaptive computing1

plugin control service

dependency management

state management states

configuration manifest

moab web services

grails plugins

single plugin type possible