Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. Spring Roo 1.0.0: Technical Deep Dive Roo Internals and Add-On Development Ben Alex, Project Lead, Spring Roo
May 17, 2015
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited.
Spring Roo 1.0.0:Technical Deep Dive
Roo Internals and Add-On Development
Ben Alex, Project Lead, Spring Roo
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 2
Agenda
• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 3
Agenda
• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 4
Design Goals
• High productivity for Java developers– Reuse their existing knowledge, skills and experience
• Eliminate barriers to adoption– Lock-in, runtime, size, development experience
• Embrace the strengths of Java– Dev-time: tooling, popularity, API quality, static typing– Deploy-time: performance, memory use, footprint
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 5
High-Impact Decisions
• Use Java– This is a tool for Java developers– It defeats the purpose if we insist on a new language
• Don't create a runtime– This avoids bugs, lock-in, adoption barriers, approvals– This avoids CPU cost, RAM cost and footprint cost
• This easily led to some sort of generation model
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 6
Prototyped Approaches
• Pluggable Annotation Processing API (JSR 269)• Generate source at build time (XDoclet-style)• Generate bytecode at build time (ASM)• Generate bytecode at runtime (ASM)• Advanced proxy-based approaches (Spring AOP)• IDE plugins
• None of them satisfied all of the design goals
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 7
Decision 1: Use AspectJ
• AspectJ ITDs for “active” generation– Active generation automatically maintains output
• Delivers compilation unit separation of concern– Easier for users, and easier for us as developers
• Instant IDE support– Reduce time to market and adoption barriers
• Other good reasons– Mature, “push in” refactor, compile-time is welcome
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 8
ITD Model
• Roo owns *_Roo_*.aj files– Will delete them if necessary
• Every ITD provider registers a “suffix” it uses– Eg “Entity” becomes *_Roo_Entity.aj– A missing ITD provider causes AJ file removal
• ITDs have proper import management– So they look and feel normal to developers– So they “push in refactor” in a natural form
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 9
Decision 2: Create A Shell
• Need something to host the “active” generation
• Need something to receive user commands
• Usability issues were of extremely high priority
• We chose an interactive shell– Highest usability (tab, context awareness, hiding, hints...)– Background monitoring of externally-made changes– Background monitoring avoided crude “generate” steps
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 10
Shell Model
• Shell will perform a “startup” scan– To handle changes made while it wasn't running
• Shell will monitor file system once started– To handle changes made while it is running
• Shell will have a set of commands registered– To handle explicit directives from the user
• Roo will never modify a *.java file except at the explicit request of a user via a shell command
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 11
That Left One Key Question...
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 12
Control Information
AspectJ ITDs
Person_Roo_ToString.aj
toString():String
Person.java
name:String
write
Person_Roo_JavaBean.aj
getName():StringsetName(String):void
readRooAdd-OnsRoo
Add-OnsRooAdd-OnsRoo
Add-Ons
Person.class
name:StringtoString():String
getName():StringsetName(String):void
AspectJCompiler
read read
writeHow will add-ons knowwhat to generate?
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 13
Metadata
• Project details– Project name, dependencies, build system...
• Java type details– Fields, methods, constructors, inheritance...
• Higher-level concepts– Entity properties, controller paths, validation rules...
• Must be automatically determined from project– With @Roo* (source level retention) where infeasible
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 14
Metadata Model
• Immutable• String-based keys (start with “MID:”)• Built on demand only (never persisted)
• Metadata can depend on other metadata– If “upsteam” metadata changes, “downstream” is told– Some metadata will want to monitor the file system
• Central metadata service and cache required
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 15
Demo #1
Shell Operation and ITD Maintenance
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 16
Agenda
• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 17
Conventions We Follow
• Ensure usability is first-class• Minimize the JAR footprint that Roo requires• Dependencies only from Enterprise Bundle Repo• Relocate runtime needs to sister Spring projects• Embrace immutability as much as possible• Maximize performance in generated code• Minimize memory consumption in generated code• Use long artifact IDs to facilitate identification• Don't put into @Roo* what you could calculate• Don't violate generator predictability conventions
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 18
The Roo Distribution
• “Roo” is split into two logical components
• Base add-ons– org.springframework.roo.addon.*– Provide major out-of-the-box Roo features– Provide metadata for other add-ons
• Roo Core– org.springframework.roo.* (excluding .addon.*)– Provide services add-ons will almost always require
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 19
1.0.0 Base Add-Ons
• Backup• Bean Info• Configurable• Data On Demand• Email• Entity• Dynamic Finder• Java Bean• JMS• JPA
• Logging• Maven• Pluralization• Property Editor• Property File• Security• Integration Test• ToString• Web (various)
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 20
Role of Third Party Add-Ons
• 3rd party add-ons have a long-term role in Roo
• Minimize download size of Roo distribution• Avoid licensing issues with Roo distribution• Facilitate features of more niche interest• Separate project management and maintenance• Enable end user customization• Provide a sandbox for easy experimentation
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 21
Core Components
Project
Support(used by all)
Metadata File MonitorModel
Classpath
File Undo Shell
Process Manager
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 22
File Monitor API
• Mostly used by project and classpath modules– Rare for an add-on to directly use file monitor
• Publishes startup and “on change” events
• Events– Create– Update– Delete– Rename (if implementation supported)– Monitoring start (for each file when first found)
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 23
File Monitor Implementation
• Polling-based implementation
• Auto-scales to reduce CPU overhead
• Shell commands– “poll status” indicates scan frequency and duration→
– “poll speed” allows specification of frequency→
– “poll now” forces an immediate poll→
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 24
Metadata Identifiers (MIDs)
• Immutable strings to avoid GC cost
• Note the “instance” portion denotes a Java type– Most MIDs include this physical location and type name– Use PhysicalTypeIdentifier.getPath(..) and
getJavaType(..) to directly extract typesafe objects
MID:o.s.r.classpath.PhysicalTypeId#SRC_MAIN_JAVA?com.foo.Bar
Static MetadataIdentificationUtils.getMdClass(..) MetadataIdentificationUtils.getMdInstance(..)
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 25
Add-On
Metadata Facilities
MetadataItem
stores
queries
MetadataProvider
registers + notifies
• MetadataService can provide all metadata in the system
• MetadataService auto-evicts all downstream dependencies
• Dependency registration can be MID class or MID instance specific
MetadataDependency
Registry
MetadataCache
MetadataService
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 26
Project Abstraction
• Abstracts the user's project build system– Immutable metadata (including dependency details)– Path (eg SRC_MAIN_JAVA) mapping to directories– Operations object to modify project structures– Access to the PathResolver instance
• Maven implementation via add-on– Designed to also support Ivy in due course (ROO-91)
• Add-ons will regularly use ProjectOperations
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 27
Process Manager
• Ensures non-overlapping synchronization– Background file monitor polling thread– Shell command foreground thread
• Facilitates “atomic” operations with rollback– Notifies disk changes until disk is in stable state– Rolls back disk changes if from a failed shell command
• Provides a FileManager abstraction– Add-ons must use this abstraction for disk changes– Reads via FileManager are optional (but recommended)
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 28
Classpath API
PhysicalType
Identifier
MID:o.s.r.classpath.PhysicalTypeId#SRC_MAIN_JAVA?com.foo.Bar
PhysicalType
Metadata
PhysicalType
Category
PhysicalType
Details
ClassOrInterfaceTypeDetails ItdTypeDetails
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 29
Classpath Implementation
• JavaParser– Provides a javacc-based AST, but no binding API– Used due to licensing considerations of alternatives
• Roo offers binding, including parameterization
• JavaParser does require source code– Avoids necessity of having a compiler running– Abstractions designed to provide ASM-like *.class
introspection should it be desired in the future
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 30
Shell
• Implementations for STS and JLine
• Add-ons should– Make a “Command” class and implement CommandMaker– Delegate methods through to an “Operations” object– Annotate methods with @CliCommand– Annotate method parameters with @CliOption– Optionally use @CliAvailabilityIndicator if desired– Throw any exceptions to abort and rollback changes– Use JDK logging or return objects for console output
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 31
Building From Source
• We develop against a public Subversion repo– You can anonymously checkout the code
• There's also a public FishEye instance (plus Jira)
• Roo itself uses Maven, so it's very easy to build– Standard package, install, assembly and site goal
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 32
Mainline Development
• Checkout Roo anonymously
svn co https://anonsvn.springframework.org/svn/spring-roo/
cd spring-roo/trunk
less readme.txt
• readme.txt contains latest setup details– Add ROO_CLASSPATH_FILE variable– Create symbolic link to bootstrap/roo-dev
• It works under Windows as well, but not as well
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 33
Classpath File Maintenance
• Classpath file allows rapid execution of the shell without loading Maven to build a full classpath
• Refreshing the ROO_CLASSPATH_FILE is easy
cd /home/spring-roo/trunk
mvn clean eclipse:clean eclipse:eclipse compile
• Use the “props” shell command to verify classpath– Use the above command to update the classpath file
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 34
Development Tips
• Load Roo using “roo-dev” (not “roo”)– Uses incremental builds as per ROO_CLASSPATH_FILE
• Note roo-dev add-ons will be under ~/roo-dev
• roo-dev always starts up in “development mode”
• Debugging is supported via Eclipse/STS– Refer to readme.txt for details
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 35
Demo #2
Setting Up A Fresh Checkout
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 36
Agenda
• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 37
Development Mode
• Enable via– Loading Roo via roo-dev shell script– JVM property developmentMode = true– Typing “development mode” at the roo> shell prompt
• Outputs full exception traces within Roo– Ugly for normal usage (exceptions are the normal way
that an add-ons abort from unexpected conditions)
• First line of troubleshooting add-on bugs is to ask for “development mode” exception data
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 38
Metadata For Type
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 39
Metadata For Id
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 40
Metadata Trace
• Use --level 1 for instance-specific notifications• Use --level 2 for absolutely all notifications
• Consider the number of notifications involved in adding a field to an entity that also has a JSP– File system change physical type metadata entity → →
metadata finder metadata bean info metadata → → →controller metadata jsp metadata→
– This doesn't even consider parallel and unchanged paths (eg integration test, getter/setters, menus etc)
– Metadata beyond type introspection is really useful!
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 41
Demo #3
Development Mode Demonstration
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 42
Agenda
• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 43
Roo's Add-On Model
• Roo 1.0.0 offers third-party add-on support
• Very simple single classloader model
• Add-on ZIPs placed in $ROO_HOME/add-ons• JARs auto-extracted into $ROO_HOME/work
• Launch scripts add $ROO_HOME/work to classpath
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 44
User Add-On Installation
• “addon install --url file:/some.ZIP”• “addon install --url http://www.x.com/s.ZIP”• “addon uninstall --pattern some*.zip”
• Install/uninstall always does a “clean” afterwards
• “addon clean”– Manages $ROO_HOME/work as appropriate– Will request to restart shell if any changes are made– Deletions are deferred until JVM shuts down
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 45
Add-On ZIP Structure
• ZIP can be created whatever way you like– Although Roo offers a Maven assembly.xml
• Directories required– “dist” contains 1+ JARs produced by an add-on– “lib” contains 0+ JARs required by the add-on– “lib” contains all transitively-necessary dependencies– Roo does not perform transitive dependency resolution
• By convention there's a readme and “legal” dir
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 46
Add-On Naming Guidelines
• Artifact IDs must be– Fully qualified (eg com.foo.bar)– Start with .com or .net or .org (due to class scanning)– End with .roo.addon (due to class scanning)
• Results in useful ZIP and JAR names like– com.my.proj.roo.addon-1.0.0.zip– com.my.proj.roo.addon-1.0.0.jar
• Roo add-on commands enforce the above– Plus offer suggestions, so don't worry too much
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 47
Add-On Creation
• It's seriously this easy:
roo> project --topLevelPackage com.my.proj.roo.addon --template ROO_ADDON_SIMPLE
roo> perform eclipse
roo> perform assembly
optional, or use mvn directly
or use Maven directly
or create a project by hand
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 48
ROO_ADDON_SIMPLE
• Depends on currently-running Roo version– Provides access to full Roo core and all base add-ons
• Creates Java types for you– These are usable types that show key Roo APIs
• Outputs a valid Maven POM and assembly.xml
• Also gives you shell scripts for classpath control
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 49
Add-On
Commands and Operations
Operations
Static FieldOptions
Operations
Static FieldOptions
ShellConverter
Command
ShellAPI
Command
Roo APIs
rego
rego
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 50
Development-Time Notes
• roo-dev uses ~/roo-dev as $ROO_HOME• roo-dev uses $ROO_ADDON_CLASSPATH_FILE
• This allows you to incrementally develop in Eclipse and not have to make a ZIP every time
• Set $ROO_ADDON_CLASSPATH_FILE via– *nix: “source /your/add-on/setup.env”– Windows: “\your\add-on\setup.bat”– Files automatically updated during “mvn compile”
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 51
Add-On Distribution
• Suggestions– Host ZIPs on your own server or Google Code SVN– Commiting to SVN simplifies HTTP distribution
• Announce via– A Tweet including #roo– A forum post at http://forum.springsource.org/– Email [email protected] with details
• I'll add it to the official list of public Roo add-ons
• A future release will revisit distribution facilities
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 52
Suggested Add-On Steps
• Learn Roo conventions from a user's perspective
• Use “project --template ROO_ADDON_SIMPLE”– Then “addon install” and try the “welcome” commands– Import it into Eclipse, study the code and customize
• Use Roo SVN to study more advanced add-ons
• Obtain help via the Spring community forums
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 53
Demo #4
Building An Add-On
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 54
Agenda
• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 55
Roo's Design
• Result of considerable R&D
• Very lightweight and loosely-coupled core
• Easy to build from SVN and round-trip develop
• “Development mode” eases troubleshooting
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 56
A Growing Ecosystem
Roo Distribution
Roo Core Base Add-Ons
Free-TextSearch Reporting Scheduling
OtherSpring
projects
DTOs/Mappers
DatabaseDeployment Captcha
HundredsMore...
Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 57
Thanks
• Hope you've enjoyed our “deep dive” into Roo!
• Roo community resources– Home → http://www.springsource.org/roo– Support → http://forum.springframework.org– Issues → http://jira.springframework.org/browse/ROO– Tweets → #roo hash key
• Questions?