33rd degree talk: open and automatic coding conventions with walkmod

Platinum Sponsor

OPEN AND AUTOMATIC CODING CONVENTIONS WITH WALKMOD

Raquel Pau

Sponsored talk by:

hello!

@raquelpau @acoroleu

which is our motivation?

DRYDon’t repeat yourself.

What are coding conventions?

“coding conventions are a set of guidelines for a specific programming language that recommend programming style, practices and methods for

each aspect of a piece program written in this language.” wikipedia

convention types

• Formatting rules: headers, indentation, documentation, code style..

• Naming rules: labeling rules for methods, classes..

• Best practices: software patterns, common bugs resolution..

google use conventions!

everybody follow conventions

http://sideeffect.kr/popularconvention

“it takes a while to create nothing”

Ron Jeffries

our goals

• automatizing the resolution of problems detected by code quality tools (i.e PMD, , sonar, findbugs).

• automatizing the development and repetitive tasks i.e: creating basic CRUD services for the entire model.

walkmod is open!

• open source: feel free to suggest changes!

• free for everybody: tell it to your boss!

• extensible by plugins do it yourself and share them! DIY+S

Introduction

walkmod is ready for Java 8

but also could by polyglot!

how it works

making awesome things happen!

>_ man walkmod

• >_ walkmod apply modifies the source code according a set of coding conventions

• >_ walkmod check specifies which source files do not follow a set of coding conventions

how it works

conventions driven by transformations

how it works

• Coding conventions are applied with blocks of transformations for each source file.

• Transformations may update the same sources or creating/updating another ones.

overview

• reader: reads the sources. i.e retrieves all files from a directory recursively.

• walker: executes a chain of transformations for each source.

• writer: writes the sources. i.e using the eclipse formatter.

how it works

how it works

reader

• Reads the sources. By default, reads the folder src/main/java.

• Works with multiple include/exclude rules.

• Creates a resource object, whose content is iterated from the walker.

how it works

walker• Executes a chain of transformations for each object allocated in a

resource. i.e all java source files of an specific folder.

• merges the output produced by transformations with existent resources.

• analyzes and reports which changes have been produced by the chain of transformations in each object.

• invokes the writer with the final (and merged) output if there is any change.

how it works

writer

• writes each object allocated in a resource. i.e all java source files of a specific folder.

• Has include/exclude rules.

• There are useful writer implementations, such as the storage of the contents of a toString() object method or the eclipse formatter.

how it works

transformations

Code conventions are a reality by means of code transformations

modify or generate code without changing walkmod

transformations

visitors

transformations

public class HelloVisitor extends VoidVisitor<VisitorContext>{...@Overwritepublic void visit(MethodDeclaration md, VisitorContext ctx){//TODO

}

@Overwritepublic void visit(FieldDeclaration fd, VisitorContext ctx){//TODO

}...

}

transformations

why visitors?

• Visitor design pattern is a way of separating an algorithm from an object structure on which it operates.

• Straight-forward way to design code transformations in java.

• Visitors can just be shared by means of walkmod plugins.

transformations

scripts

transformations

transformations

scripts

• Scripts allow the design of inline transformations.

• Scripts should be used to apply simple modifications in source files.

• Support for multiple languages. Those which implement the standard Java scripting interface. i.e. groovy, javascript, python..

transformations

templates

transformations

transformations

templatestransformations

• What You See Is What You Get (WYSIWYG): Simplest way to dynamic content to follow a given convention.

• groovy is the default template engine, but can be customized.

remembertransformations

• Visitors: a straight-forward way to design code transformations in java.

• scripts: to design inline transformations without creating a java library.

• templates: WYSIWYG transformations.

query engine

Write less to do the same!

query engine

MethodDeclaration method = null;

Collection members = type.getMembers();

Iterator it = members.iterator();

while (it.hasNext()){

BodyDeclaration member = (BodyDeclaration)it.next();

if (member instance of MethodDeclararion){

MethodDeclarion aux = (MethodDeclaration) member;

if(“execute”.equals(aux.getName()){

method = aux;

break;

}

}

}

type.methods.find({it.name.equals(“execute”)})

{ context } + query expression

query engine

query language

• The default query language is gPath (groovy), but you can change it for your favorite language.

• Common used large query expressions can be referenced from Alias. “TypeDeclaration.metaClass.getMethods = { -> delegate.members.findAll({it instanceof MethodDeclaration}); }”

query engine

queries from templates

Using the query object: ${query.resolve(“expr”)}.

query engine

import org.apache.log4j.Logger;

public class ${query.resolve("type.name")}{

public static Logger log = Logger.getLogger(${query.resolve("type.name")}.class);

}

template to add Loggers

queries from scripts

accessing through a binding called query

query engine

..

for( type in node.types) {

def result = query.resolve(type, “methods”);

...

}

...

groovy script querying the type methods

queries from visitors

Implementing QueryEngineAware or extending VisitorSupport.

query engine

public class MyVisitor extends VisitorSupport{

@Overwrite

public void visit(TypeDeclaration td, VisitorContext ctx){

Object result = query(

td, //context

“methods.find({it.name.equals(“foo”)})” //expr

);

}

}

visitor code with a gpath query

merge engine

Don’t loose your changes

why a merge engine?

• Developer changes are respected (e.g. adding a new method).

• Simplify transformations. Otherwise, transformations would check many conditions to avoid repeating code or overwriting it.

merge engine

merge engine

merge engine

merge engine

merge engine

merge engine

semantic merge

• Code is merged according to the meaning of its elements instead of simply merging text.

• Each node type has an specific merge policy.

merge engine

named nodes | anonymous nodes

• named nodes: those nodes identifiable inside an AST.

e.g. methods or fields of a given type

• anonymous nodes: those non-identifiable.

e.g. statements of a method

merge engine

merging named nodes

• append policy: only adds non existing nodes generated by a transformation.

• overwrite policy: always replaces existing nodes by the generated ones.

merge engine

merging anonymous nodes

• assign policy replaces the whole original list of nodes by generated nodes.

• unmodify policy maintains the original list of nodes.

• addall policy appends the whole list of generated nodes into the original list.

merge engine

plugins

Let’s grow up!

walkmod is growing up

plugins

where can I find them?plugins

how to extend walkmod?

• Creating new java libraries and deploying them into a maven repository (public or private) with the name walkmod-xxx-plugin.

• All walkmod extensions need a plugin descriptor of their components in the META-INF/walkmod directory inside the jar library.

plugins

plugins backend

• Walkmod embeds apache ivy to download plugins from maven repositories in runtime.

• Custom repositories are in ${walkmod-home}/conf/ivysettings.xml

• All plugin jars and their dependencies are files loaded dynamically into a new classloader.

plugins

roadmap

next steps

• + plugins: Create and publish new plugins (e.g. naming rules or support for other programming languages).

• - less configuration: reutilization by inheritance / include rules of the XML elements.

• saas: publish online services for running walkmod and all its plugins (also private).

roadmap

thanks for your time!

and.. do cool things that matter ;)

@walkmod